Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-09-26T19:58:19Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:58:19Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260926 19:58:19.736863 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.254:33143
--webserver_interface=127.11.233.254
--webserver_port=0
--builtin_ntp_servers=127.11.233.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 19:58:20.055936 12208 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:58:20.056569 12208 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:58:20.057019 12208 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:58:20.092958 12208 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:58:20.093309 12208 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:58:20.093614 12208 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:58:20.093883 12208 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:58:20.131707 12208 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.254:33143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.233.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:58:20.133102 12208 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:58:20.134832 12208 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:58:20.146364 12214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:20.147357 12215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:21.357762 12217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:21.362143 12216 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1213 milliseconds
W20260926 19:58:21.363059 12208 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.216s user 0.412s sys 0.802s
W20260926 19:58:21.363377 12208 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.217s user 0.412s sys 0.802s
I20260926 19:58:21.363664 12208 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:58:21.364848 12208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:58:21.367782 12208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:58:21.369223 12208 hybrid_clock.cc:648] HybridClock initialized: now 1790452701369170 us; error 66 us; skew 500 ppm
I20260926 19:58:21.370162 12208 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:58:21.377554 12208 webserver.cc:533] Webserver started at http://127.11.233.254:33189/ using document root <none> and password file <none>
I20260926 19:58:21.378589 12208 fs_manager.cc:362] Metadata directory not provided
I20260926 19:58:21.378856 12208 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:58:21.379321 12208 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:58:21.386013 12208 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/data/instance:
uuid: "db9d871438324598b27dd67b31c017d9"
format_stamp: "Formatted at 2026-09-26 19:58:21 on dist-test-slave-jcdj"
I20260926 19:58:21.387295 12208 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/wal/instance:
uuid: "db9d871438324598b27dd67b31c017d9"
format_stamp: "Formatted at 2026-09-26 19:58:21 on dist-test-slave-jcdj"
I20260926 19:58:21.395546 12208 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s
I20260926 19:58:21.402334 12224 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:21.403707 12208 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260926 19:58:21.404074 12208 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/wal
uuid: "db9d871438324598b27dd67b31c017d9"
format_stamp: "Formatted at 2026-09-26 19:58:21 on dist-test-slave-jcdj"
I20260926 19:58:21.404448 12208 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:58:21.482579 12208 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:58:21.484108 12208 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:58:21.484584 12208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:58:21.556077 12208 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.254:33143
I20260926 19:58:21.556146 12275 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.254:33143 every 8 connection(s)
I20260926 19:58:21.559198 12208 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
I20260926 19:58:21.562624 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 12208
I20260926 19:58:21.563181 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-0/wal/instance
I20260926 19:58:21.566385 12276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:21.568607 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.253:37913
--webserver_interface=127.11.233.253
--webserver_port=0
--builtin_ntp_servers=127.11.233.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 19:58:21.579438 12276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:58:21.609468 12276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } has no permanent_uuid. Determining permanent_uuid...
W20260926 19:58:21.612106 12227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
W20260926 19:58:21.616535 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
I20260926 19:58:21.666044 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } attempt: 1
W20260926 19:58:21.670666 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
I20260926 19:58:21.739171 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } attempt: 2
W20260926 19:58:21.743528 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
I20260926 19:58:21.835000 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } attempt: 3
W20260926 19:58:21.839452 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
W20260926 19:58:21.885802 12278 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:58:21.886351 12278 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:58:21.886718 12278 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:58:21.920536 12278 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:58:21.920835 12278 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:58:21.921051 12278 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:58:21.921248 12278 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:58:21.958652 12278 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.253:37913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.233.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:58:21.960069 12278 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:58:21.961705 12278 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:58:21.972153 12287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:21.982955 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } attempt: 4
W20260926 19:58:21.989787 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
I20260926 19:58:22.289340 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } attempt: 5
W20260926 19:58:22.297206 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
I20260926 19:58:22.844817 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } attempt: 6
W20260926 19:58:22.872686 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
W20260926 19:58:23.376368 12286 debug-util.cc:398] Leaking SignalData structure 0x7b0800034de0 after lost signal to thread 12278
W20260926 19:58:23.417574 12278 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.444s user 0.511s sys 0.929s
W20260926 19:58:23.418084 12278 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.445s user 0.511s sys 0.930s
W20260926 19:58:21.973855 12288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:23.419389 12289 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1444 milliseconds
W20260926 19:58:23.420480 12290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:23.420411 12278 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:58:23.423599 12278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:58:23.426152 12278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:58:23.427500 12278 hybrid_clock.cc:648] HybridClock initialized: now 1790452703427465 us; error 66 us; skew 500 ppm
I20260926 19:58:23.428305 12278 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:58:23.434372 12278 webserver.cc:533] Webserver started at http://127.11.233.253:36963/ using document root <none> and password file <none>
I20260926 19:58:23.435374 12278 fs_manager.cc:362] Metadata directory not provided
I20260926 19:58:23.435621 12278 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:58:23.436082 12278 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:58:23.440397 12278 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/instance:
uuid: "db2490adb6e84090b780f52470e580ab"
format_stamp: "Formatted at 2026-09-26 19:58:23 on dist-test-slave-jcdj"
I20260926 19:58:23.441496 12278 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal/instance:
uuid: "db2490adb6e84090b780f52470e580ab"
format_stamp: "Formatted at 2026-09-26 19:58:23 on dist-test-slave-jcdj"
I20260926 19:58:23.448931 12278 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260926 19:58:23.454456 12299 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:23.455541 12278 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260926 19:58:23.455865 12278 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
uuid: "db2490adb6e84090b780f52470e580ab"
format_stamp: "Formatted at 2026-09-26 19:58:23 on dist-test-slave-jcdj"
I20260926 19:58:23.456204 12278 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:58:23.508471 12278 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:58:23.510104 12278 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:58:23.510566 12278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:58:23.584324 12278 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.253:37913
I20260926 19:58:23.584407 12350 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.253:37913 every 8 connection(s)
I20260926 19:58:23.587138 12278 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
I20260926 19:58:23.588626 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 12278
I20260926 19:58:23.589203 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal/instance
I20260926 19:58:23.592041 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.252:44243
--webserver_interface=127.11.233.252
--webserver_port=0
--builtin_ntp_servers=127.11.233.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 19:58:23.593945 12351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:23.604173 12351 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:58:23.625789 12351 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:58:23.640228 12351 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } has no permanent_uuid. Determining permanent_uuid...
W20260926 19:58:23.642694 12303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
W20260926 19:58:23.647037 12351 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:23.696617 12351 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 1
W20260926 19:58:23.701689 12351 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:23.770125 12351 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 2
W20260926 19:58:23.774510 12351 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:23.865923 12351 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 3
W20260926 19:58:23.870394 12351 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
W20260926 19:58:23.911890 12353 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:58:23.912513 12353 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:58:23.912945 12353 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260926 19:58:23.933359 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } attempt: 7
I20260926 19:58:23.944680 12276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } has no permanent_uuid. Determining permanent_uuid...
W20260926 19:58:23.948060 12353 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:58:23.948473 12353 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:58:23.948771 12353 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:58:23.949035 12353 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260926 19:58:23.950479 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:23.986582 12353 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.252:44243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.233.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:58:23.987871 12353 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:58:23.989488 12353 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:58:24.000393 12365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:24.008908 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 1
I20260926 19:58:24.014050 12351 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 4
W20260926 19:58:24.021871 12351 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
W20260926 19:58:24.022924 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:24.104503 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 2
W20260926 19:58:24.113252 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:24.201005 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 3
W20260926 19:58:24.209875 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:24.321468 12351 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 5
W20260926 19:58:24.330094 12351 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:24.350596 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 4
W20260926 19:58:24.359591 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:24.643162 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 5
W20260926 19:58:24.652153 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:24.877686 12351 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 6
W20260926 19:58:24.906540 12351 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
I20260926 19:58:25.204756 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 6
W20260926 19:58:25.233799 12276 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:44243: Network error: Client connection negotiation failed: client connection to 127.11.233.252:44243: connect: Connection refused (error 111)
W20260926 19:58:24.000797 12366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:25.296892 12368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:25.298897 12367 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1293 milliseconds
W20260926 19:58:25.300100 12353 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.299s user 0.474s sys 0.815s
W20260926 19:58:25.300375 12353 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.300s user 0.474s sys 0.815s
I20260926 19:58:25.300562 12353 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:58:25.301723 12353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:58:25.304378 12353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:58:25.305758 12353 hybrid_clock.cc:648] HybridClock initialized: now 1790452705305730 us; error 42 us; skew 500 ppm
I20260926 19:58:25.306583 12353 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:58:25.313997 12353 webserver.cc:533] Webserver started at http://127.11.233.252:46287/ using document root <none> and password file <none>
I20260926 19:58:25.315050 12353 fs_manager.cc:362] Metadata directory not provided
I20260926 19:58:25.315302 12353 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:58:25.315763 12353 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:58:25.320194 12353 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/data/instance:
uuid: "8a69f5870f8d461aa5e61f6999bb05c6"
format_stamp: "Formatted at 2026-09-26 19:58:25 on dist-test-slave-jcdj"
I20260926 19:58:25.321383 12353 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/wal/instance:
uuid: "8a69f5870f8d461aa5e61f6999bb05c6"
format_stamp: "Formatted at 2026-09-26 19:58:25 on dist-test-slave-jcdj"
I20260926 19:58:25.329931 12353 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.002s sys 0.004s
I20260926 19:58:25.336102 12379 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:25.337352 12353 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260926 19:58:25.337735 12353 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/wal
uuid: "8a69f5870f8d461aa5e61f6999bb05c6"
format_stamp: "Formatted at 2026-09-26 19:58:25 on dist-test-slave-jcdj"
I20260926 19:58:25.338109 12353 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:58:25.419097 12353 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:58:25.420656 12353 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:58:25.421146 12353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:58:25.503836 12353 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.252:44243
I20260926 19:58:25.503917 12430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.252:44243 every 8 connection(s)
I20260926 19:58:25.506670 12353 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/data/info.pb
I20260926 19:58:25.509368 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 12353
I20260926 19:58:25.509951 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-2/wal/instance
I20260926 19:58:25.513397 12431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:25.537117 12431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:58:25.563519 12431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:58:25.575531 12431 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:58:25.595945 12431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Bootstrap starting.
I20260926 19:58:25.602231 12431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:25.604140 12431 log.cc:826] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Log is configured to *not* fsync() on all Append() calls
I20260926 19:58:25.610381 12431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: No bootstrap required, opened a new log
I20260926 19:58:25.631681 12431 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:25.632381 12431 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:25.632647 12431 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a69f5870f8d461aa5e61f6999bb05c6, State: Initialized, Role: FOLLOWER
I20260926 19:58:25.633396 12431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:25.635758 12440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:25.636897 12440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:25.637621 12431 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:58:25.652872 12440 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:25.653393 12440 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:25.657205 12440 leader_election.cc:290] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), db2490adb6e84090b780f52470e580ab (127.11.233.253:37913)
W20260926 19:58:25.668061 12451 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:58:25.668460 12451 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:58:25.677122 12385 tablet.cc:2433] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:58:25.664217 12353 master_runner.cc:187] Error getting master registration for 127.11.233.254:33143: OK, instance_id { permanent_uuid: "db9d871438324598b27dd67b31c017d9" instance_seqno: 1790452701536309 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:58:25.696214 12353 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:58:25.967072 12351 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 7
I20260926 19:58:25.992359 12351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Bootstrap starting.
I20260926 19:58:26.002908 12351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:26.005287 12351 log.cc:826] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Log is configured to *not* fsync() on all Append() calls
I20260926 19:58:26.011536 12351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: No bootstrap required, opened a new log
I20260926 19:58:26.041721 12351 raft_consensus.cc:359] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:26.042639 12351 raft_consensus.cc:385] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:26.042950 12351 raft_consensus.cc:740] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2490adb6e84090b780f52470e580ab, State: Initialized, Role: FOLLOWER
I20260926 19:58:26.043947 12351 consensus_queue.cc:260] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:26.047991 12455 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:26.049482 12455 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:26.049871 12351 sys_catalog.cc:565] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:58:26.050052 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:58:26.050935 12326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 0.
I20260926 19:58:26.052943 12382 leader_election.cc:304] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab; no voters:
I20260926 19:58:26.054026 12440 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:26.054363 12440 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:26.054682 12440 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:26.061589 12440 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:26.069901 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab"
I20260926 19:58:26.070483 12326 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:26.073822 12440 leader_election.cc:290] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 1 election: Requested vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), db2490adb6e84090b780f52470e580ab (127.11.233.253:37913)
I20260926 19:58:26.079113 12326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 1.
I20260926 19:58:26.080328 12382 leader_election.cc:304] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab; no voters:
I20260926 19:58:26.081168 12440 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:26.083442 12440 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 1 LEADER]: Becoming Leader. State: Replica: 8a69f5870f8d461aa5e61f6999bb05c6, State: Running, Role: LEADER
I20260926 19:58:26.084537 12440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:26.100698 12461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 1 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:26.105893 12461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
W20260926 19:58:26.128890 12468 catalog_manager.cc:1607] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 19:58:26.129073 12469 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
W20260926 19:58:26.129350 12468 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:58:26.152020 12469 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 19:58:26.112150 12278 master_runner.cc:187] Error getting master registration for 127.11.233.254:33143: OK, instance_id { permanent_uuid: "db9d871438324598b27dd67b31c017d9" instance_seqno: 1790452701536309 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:58:26.166337 12278 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:58:26.181164 12326 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Refusing update from remote peer 8a69f5870f8d461aa5e61f6999bb05c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:26.187112 12461 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:26.234012 12471 mvcc.cc:204] Tried to move back new op lower bound from 7333694284496887808 to 7333694284150312960. Current Snapshot: MvccSnapshot[applied={T|T < 7333694284496887808}]
I20260926 19:58:26.233268 12455 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 1 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:26.241722 12455 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:26.258129 12440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:26.258952 12440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:26.267829 12469 catalog_manager.cc:1396] Generated new cluster ID: 74efea6f15be4c5bb4893708cb04c269
I20260926 19:58:26.268172 12469 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:58:26.269181 12276 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } attempt: 7
W20260926 19:58:26.290552 12381 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.233.254:33143 timed out after 0.634s (SENT)
I20260926 19:58:26.285457 12455 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:26.301930 12455 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:26.322932 12469 catalog_manager.cc:1419] Generated new certificate authority record
I20260926 19:58:26.324990 12469 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:58:26.333303 12276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9: Bootstrap starting.
I20260926 19:58:26.340719 12276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:26.343551 12276 log.cc:826] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9: Log is configured to *not* fsync() on all Append() calls
I20260926 19:58:26.365097 12276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9: No bootstrap required, opened a new log
I20260926 19:58:26.368903 12469 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Generated new TSK 0
I20260926 19:58:26.370069 12469 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:58:26.408993 12276 raft_consensus.cc:359] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:26.410095 12276 raft_consensus.cc:385] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:26.410532 12276 raft_consensus.cc:740] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db9d871438324598b27dd67b31c017d9, State: Initialized, Role: FOLLOWER
I20260926 19:58:26.410663 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.233.193:0
--local_ip_for_outbound_sockets=127.11.233.193
--webserver_interface=127.11.233.193
--webserver_port=0
--tserver_master_addrs=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--builtin_ntp_servers=127.11.233.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 19:58:26.411998 12276 consensus_queue.cc:260] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:26.427805 12480 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:26.429456 12276 sys_catalog.cc:565] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:58:26.429617 12250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9"
I20260926 19:58:26.430673 12249 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:26.432659 12248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:26.433724 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
W20260926 19:58:26.435072 12251 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40766 (request call id 1) took 778 ms (client timeout 634 ms). Trace:
W20260926 19:58:26.435432 12251 rpcz_store.cc:269] 0926 19:58:25.656626 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:25.656768 (+ 142us) service_pool.cc:224] Handling call
0926 19:58:26.434992 (+778224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:58:26.442058 12249 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Refusing update from remote peer 8a69f5870f8d461aa5e61f6999bb05c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:26.444238 12440 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:26.456595 12480 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:26.520835 12480 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:26.520939 12483 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 1 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:26.521615 12480 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:26.521889 12483 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:26.552412 12497 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260926 19:58:26.903995 12481 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:58:26.904939 12481 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:58:26.905604 12481 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:58:26.940567 12481 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:58:26.941601 12481 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.233.193
I20260926 19:58:26.952564 12483 raft_consensus.cc:493] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:26.953168 12483 raft_consensus.cc:515] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:26.957199 12325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:58:26.959739 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:26.963392 12228 leader_election.cc:304] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db9d871438324598b27dd67b31c017d9; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab
I20260926 19:58:26.964303 12480 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 19:58:26.955684 12483 leader_election.cc:290] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db2490adb6e84090b780f52470e580ab (127.11.233.253:37913), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:26.987985 12481 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.233.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.233.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.233.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:58:26.989464 12481 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:58:26.991401 12481 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:58:27.006660 12503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:27.096213 12509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:27.101773 12509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:27.136574 12468 catalog_manager.cc:1308] Loaded cluster ID: 74efea6f15be4c5bb4893708cb04c269
I20260926 19:58:27.137169 12468 catalog_manager.cc:1601] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: loading cluster ID for follower catalog manager: success
I20260926 19:58:27.157799 12468 catalog_manager.cc:1623] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: acquiring CA information for follower catalog manager: success
I20260926 19:58:27.169250 12468 catalog_manager.cc:1651] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:58:27.651172 12497 catalog_manager.cc:1308] Loaded cluster ID: 74efea6f15be4c5bb4893708cb04c269
I20260926 19:58:27.654925 12497 catalog_manager.cc:1601] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9: loading cluster ID for follower catalog manager: success
I20260926 19:58:27.689005 12497 catalog_manager.cc:1623] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9: acquiring CA information for follower catalog manager: success
I20260926 19:58:27.723717 12497 catalog_manager.cc:1651] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:58:28.194505 12512 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:28.195299 12512 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:28.209566 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:28.212682 12512 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:28.213904 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:28.215627 12303 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:58:28.216907 12512 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 19:58:28.770195 12513 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:28.770879 12513 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:28.774847 12513 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:28.775822 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:28.777081 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:28.779918 12301 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:58:28.780771 12513 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 19:58:28.408468 12502 debug-util.cc:398] Leaking SignalData structure 0x7b0800037180 after lost signal to thread 12481
W20260926 19:58:28.787423 12481 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.777s user 0.000s sys 0.003s
W20260926 19:58:28.787438 12505 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection timed out after 1778 milliseconds
W20260926 19:58:28.788478 12481 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.778s user 0.000s sys 0.003s
W20260926 19:58:27.009227 12504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:28.789723 12507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:28.790171 12481 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:58:28.794034 12481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:58:28.797601 12481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:58:28.799085 12481 hybrid_clock.cc:648] HybridClock initialized: now 1790452708798992 us; error 100 us; skew 500 ppm
I20260926 19:58:28.799962 12481 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:58:28.819499 12481 webserver.cc:533] Webserver started at http://127.11.233.193:46299/ using document root <none> and password file <none>
I20260926 19:58:28.820847 12481 fs_manager.cc:362] Metadata directory not provided
I20260926 19:58:28.821174 12481 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:58:28.821815 12481 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:58:28.830331 12481 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/data/instance:
uuid: "f6e9d6439fd54b28925a68a6145d3dee"
format_stamp: "Formatted at 2026-09-26 19:58:28 on dist-test-slave-jcdj"
I20260926 19:58:28.832074 12481 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/instance:
uuid: "f6e9d6439fd54b28925a68a6145d3dee"
format_stamp: "Formatted at 2026-09-26 19:58:28 on dist-test-slave-jcdj"
I20260926 19:58:28.848295 12481 fs_manager.cc:696] Time spent creating directory manager: real 0.015s user 0.011s sys 0.004s
I20260926 19:58:28.862488 12518 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:28.863981 12481 fs_manager.cc:730] Time spent opening block manager: real 0.011s user 0.007s sys 0.004s
I20260926 19:58:28.864423 12481 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal
uuid: "f6e9d6439fd54b28925a68a6145d3dee"
format_stamp: "Formatted at 2026-09-26 19:58:28 on dist-test-slave-jcdj"
I20260926 19:58:28.864885 12481 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:58:28.962466 12481 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:58:28.964597 12481 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:58:28.965200 12481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:58:28.975564 12481 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 19:58:28.987566 12481 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:58:28.987884 12481 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:28.988215 12481 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 19:58:28.988442 12481 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:29.311838 12563 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:29.321797 12563 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:29.324626 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:29.327288 12563 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:29.328136 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:29.330870 12303 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:58:29.331921 12563 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 19:58:29.334169 12566 raft_consensus.cc:493] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:29.345750 12566 raft_consensus.cc:515] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:29.348903 12325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:58:29.349053 12566 leader_election.cc:290] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db2490adb6e84090b780f52470e580ab (127.11.233.253:37913), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:29.349573 12325 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db9d871438324598b27dd67b31c017d9 in term 1.
I20260926 19:58:29.352216 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:29.353915 12227 leader_election.cc:304] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db2490adb6e84090b780f52470e580ab, db9d871438324598b27dd67b31c017d9; no voters:
I20260926 19:58:29.354887 12566 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 19:58:29.355376 12566 raft_consensus.cc:493] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:29.355808 12566 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:29.363798 12566 raft_consensus.cc:515] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:29.368005 12325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab"
I20260926 19:58:29.368559 12325 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:29.375181 12325 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db9d871438324598b27dd67b31c017d9 in term 2.
I20260926 19:58:29.376528 12227 leader_election.cc:304] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db2490adb6e84090b780f52470e580ab, db9d871438324598b27dd67b31c017d9; no voters:
I20260926 19:58:29.382376 12571 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:58:29.382611 12566 leader_election.cc:290] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 2 election: Requested vote from peers db2490adb6e84090b780f52470e580ab (127.11.233.253:37913), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:29.384420 12571 raft_consensus.cc:697] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 2 LEADER]: Becoming Leader. State: Replica: db9d871438324598b27dd67b31c017d9, State: Running, Role: LEADER
I20260926 19:58:29.384146 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6"
I20260926 19:58:29.385946 12571 consensus_queue.cc:237] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:29.410791 12566 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db9d871438324598b27dd67b31c017d9. Latest consensus state: current_term: 2 leader_uuid: "db9d871438324598b27dd67b31c017d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:29.411509 12566 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:29.422614 12579 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 19:58:29.429862 12325 raft_consensus.cc:1240] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Rejecting Update request from peer 8a69f5870f8d461aa5e61f6999bb05c6 for earlier term 1. Current term is 2. Ops: []
I20260926 19:58:29.431870 12579 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 19:58:29.434672 12584 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 19:58:29.436969 12579 catalog_manager.cc:1308] Loaded cluster ID: 74efea6f15be4c5bb4893708cb04c269
I20260926 19:58:29.437422 12579 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:58:29.444272 12251 raft_consensus.cc:1240] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 2 LEADER]: Rejecting Update request from peer 8a69f5870f8d461aa5e61f6999bb05c6 for earlier term 1. Current term is 2. Ops: []
I20260926 19:58:29.445850 12576 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 1 LEADER]: Stepping down as leader of term 1
I20260926 19:58:29.446200 12576 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a69f5870f8d461aa5e61f6999bb05c6, State: Running, Role: LEADER
I20260926 19:58:29.444296 12579 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:58:29.446559 12584 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 19:58:29.447252 12584 consensus_queue.cc:1243] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Peer db9d871438324598b27dd67b31c017d9 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260926 19:58:29.447889 12576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:29.450320 12576 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:29.453886 12579 catalog_manager.cc:6230] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9: Loaded TSK: 0
I20260926 19:58:29.455199 12579 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:58:29.659147 12481 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.193:46439
I20260926 19:58:29.659276 12639 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.193:46439 every 8 connection(s)
I20260926 19:58:29.663159 12481 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/data/info.pb
I20260926 19:58:29.673236 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 12481
I20260926 19:58:29.673892 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/instance
I20260926 19:58:29.677925 12451 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:58:29.695341 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.233.194:0
--local_ip_for_outbound_sockets=127.11.233.194
--webserver_interface=127.11.233.194
--webserver_port=0
--tserver_master_addrs=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--builtin_ntp_servers=127.11.233.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 19:58:29.756227 12640 heartbeater.cc:344] Connected to a master server at 127.11.233.252:44243
I20260926 19:58:29.756742 12640 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:29.758082 12640 heartbeater.cc:507] Master 127.11.233.252:44243 requested a full tablet report, sending...
I20260926 19:58:29.762171 12396 ts_manager.cc:194] Registered new tserver with Master: f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:29.770422 12641 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
I20260926 19:58:29.770804 12641 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:29.771739 12641 heartbeater.cc:507] Master 127.11.233.253:37913 requested a full tablet report, sending...
I20260926 19:58:29.775259 12316 ts_manager.cc:194] Registered new tserver with Master: f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:29.780843 12646 heartbeater.cc:344] Connected to a master server at 127.11.233.254:33143
I20260926 19:58:29.781226 12646 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:29.782249 12646 heartbeater.cc:507] Master 127.11.233.254:33143 requested a full tablet report, sending...
I20260926 19:58:29.785180 12241 ts_manager.cc:194] Registered new tserver with Master: f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:29.788720 12241 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.11.233.193:56309
I20260926 19:58:29.853915 12325 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Refusing update from remote peer db9d871438324598b27dd67b31c017d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260926 19:58:29.856417 12566 consensus_queue.cc:1048] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 19:58:29.887075 12652 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader db9d871438324598b27dd67b31c017d9. Latest consensus state: current_term: 2 leader_uuid: "db9d871438324598b27dd67b31c017d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:29.887967 12652 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:29.899336 12566 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "db9d871438324598b27dd67b31c017d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:29.900049 12566 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:29.900876 12406 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 2 FOLLOWER]: Refusing update from remote peer db9d871438324598b27dd67b31c017d9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260926 19:58:29.902686 12571 consensus_queue.cc:1048] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 19:58:29.918844 12581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "db9d871438324598b27dd67b31c017d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:29.919586 12581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:29.920666 12576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db9d871438324598b27dd67b31c017d9. Latest consensus state: current_term: 2 leader_uuid: "db9d871438324598b27dd67b31c017d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:29.921228 12652 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "db9d871438324598b27dd67b31c017d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:29.921347 12576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:29.921940 12652 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
W20260926 19:58:30.178609 12645 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:58:30.179315 12645 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:58:30.179986 12645 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:58:30.217156 12645 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:58:30.218214 12645 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.233.194
I20260926 19:58:30.257953 12645 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.233.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.233.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.233.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:58:30.259408 12645 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:58:30.261564 12645 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:58:30.276388 12664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:30.292835 12646 heartbeater.cc:499] Master 127.11.233.254:33143 was elected leader, sending a full tablet report...
I20260926 19:58:30.395146 12652 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db9d871438324598b27dd67b31c017d9)
I20260926 19:58:30.395839 12652 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:30.401952 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:30.405576 12652 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:30.406697 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:30.406698 12576 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db9d871438324598b27dd67b31c017d9)
I20260926 19:58:30.407860 12576 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:30.409987 12576 leader_election.cc:290] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), db2490adb6e84090b780f52470e580ab (127.11.233.253:37913)
I20260926 19:58:30.411058 12406 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db2490adb6e84090b780f52470e580ab in term 2.
I20260926 19:58:30.412303 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:58:30.412315 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:30.412881 12326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 2.
I20260926 19:58:30.415297 12382 leader_election.cc:304] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab; no voters: db9d871438324598b27dd67b31c017d9
I20260926 19:58:30.416534 12303 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab; no voters: db9d871438324598b27dd67b31c017d9
I20260926 19:58:30.417331 12652 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 19:58:30.417670 12576 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 19:58:30.417800 12652 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Starting leader election (detected failure of leader db9d871438324598b27dd67b31c017d9)
I20260926 19:58:30.418004 12576 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader db9d871438324598b27dd67b31c017d9)
I20260926 19:58:30.418257 12652 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 2 FOLLOWER]: Advancing to term 3
I20260926 19:58:30.418349 12576 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 2 FOLLOWER]: Advancing to term 3
I20260926 19:58:30.425276 12576 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:30.431871 12576 leader_election.cc:290] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 3 election: Requested vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), db2490adb6e84090b780f52470e580ab (127.11.233.253:37913)
I20260926 19:58:30.433365 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab"
I20260926 19:58:30.433561 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9"
I20260926 19:58:30.429404 12652 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:30.436661 12326 raft_consensus.cc:2393] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a69f5870f8d461aa5e61f6999bb05c6 in current term 3: Already voted for candidate db2490adb6e84090b780f52470e580ab in this term.
I20260926 19:58:30.440178 12382 leader_election.cc:304] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6; no voters: db2490adb6e84090b780f52470e580ab, db9d871438324598b27dd67b31c017d9
I20260926 19:58:30.441192 12576 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260926 19:58:30.445051 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9"
I20260926 19:58:30.449518 12652 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 3 election: Requested vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:30.449968 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6"
I20260926 19:58:30.450742 12406 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db2490adb6e84090b780f52470e580ab in current term 3: Already voted for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in this term.
I20260926 19:58:30.452108 12303 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:58:30.452883 12652 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260926 19:58:30.506656 12326 raft_consensus.cc:1240] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 3 FOLLOWER]: Rejecting Update request from peer db9d871438324598b27dd67b31c017d9 for earlier term 2. Current term is 3. Ops: []
I20260926 19:58:30.520893 12670 consensus_queue.cc:1059] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260926 19:58:30.530295 12670 raft_consensus.cc:3055] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 2 LEADER]: Stepping down as leader of term 2
I20260926 19:58:30.535825 12670 raft_consensus.cc:740] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db9d871438324598b27dd67b31c017d9, State: Running, Role: LEADER
I20260926 19:58:30.537075 12406 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Rejecting Update request from peer db9d871438324598b27dd67b31c017d9 for earlier term 2. Current term is 3. Ops: []
I20260926 19:58:30.537160 12670 consensus_queue.cc:260] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:30.539661 12670 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 2 FOLLOWER]: Advancing to term 3
I20260926 19:58:30.722800 12353 master_runner.cc:264] No leader master found from master 8a69f5870f8d461aa5e61f6999bb05c6
I20260926 19:58:30.723193 12353 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:58:30.960421 12673 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:30.962544 12673 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:30.967651 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:30.968628 12251 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 3.
I20260926 19:58:30.970665 12381 leader_election.cc:304] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9; no voters:
I20260926 19:58:30.970875 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:58:30.971329 12673 leader_election.cc:290] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), db2490adb6e84090b780f52470e580ab (127.11.233.253:37913)
I20260926 19:58:30.971454 12326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 3.
I20260926 19:58:30.973582 12673 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260926 19:58:30.974752 12673 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:30.976404 12673 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 3 FOLLOWER]: Advancing to term 4
I20260926 19:58:31.000651 12673 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:31.006166 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9"
I20260926 19:58:31.007014 12251 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 3 FOLLOWER]: Advancing to term 4
I20260926 19:58:31.009130 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab"
I20260926 19:58:31.009704 12326 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 3 FOLLOWER]: Advancing to term 4
I20260926 19:58:31.009764 12673 leader_election.cc:290] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 4 election: Requested vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), db2490adb6e84090b780f52470e580ab (127.11.233.253:37913)
I20260926 19:58:31.014668 12251 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 4.
I20260926 19:58:31.016573 12326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 4.
I20260926 19:58:31.017800 12381 leader_election.cc:304] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9; no voters:
I20260926 19:58:31.018936 12673 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 4 FOLLOWER]: Leader election won for term 4
I20260926 19:58:31.020136 12673 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 4 LEADER]: Becoming Leader. State: Replica: 8a69f5870f8d461aa5e61f6999bb05c6, State: Running, Role: LEADER
I20260926 19:58:31.021716 12673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:31.035336 12674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 4 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:31.036341 12674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:31.051355 12676 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 19:58:31.068758 12676 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 19:58:31.077558 12676 catalog_manager.cc:1308] Loaded cluster ID: 74efea6f15be4c5bb4893708cb04c269
I20260926 19:58:31.085346 12676 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:58:31.097965 12676 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:58:31.116242 12676 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Loaded TSK: 0
I20260926 19:58:31.119614 12676 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:58:31.514114 12326 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Refusing update from remote peer 8a69f5870f8d461aa5e61f6999bb05c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260926 19:58:31.516842 12673 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260926 19:58:31.579144 12677 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 4 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:31.579865 12677 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:31.615347 12251 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Refusing update from remote peer 8a69f5870f8d461aa5e61f6999bb05c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260926 19:58:31.619688 12673 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 19:58:31.632304 12677 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:31.633041 12677 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:31.639447 12673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:31.640134 12673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:31.679064 12683 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 4 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:31.679858 12683 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:31.680703 12687 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:31.681632 12687 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:31.776407 12640 heartbeater.cc:499] Master 127.11.233.252:44243 was elected leader, sending a full tablet report...
W20260926 19:58:31.679971 12663 debug-util.cc:398] Leaking SignalData structure 0x7b0800037180 after lost signal to thread 12645
W20260926 19:58:30.277618 12665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:31.914894 12663 kernel_stack_watchdog.cc:198] Thread 12645 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 398ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260926 19:58:31.919827 12645 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.641s user 0.001s sys 0.004s
W20260926 19:58:31.920167 12645 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.641s user 0.001s sys 0.004s
W20260926 19:58:31.920308 12666 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1639 milliseconds
W20260926 19:58:31.921865 12668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:31.921932 12645 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:58:31.923092 12645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:58:31.925153 12645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:58:31.926524 12645 hybrid_clock.cc:648] HybridClock initialized: now 1790452711926488 us; error 36 us; skew 500 ppm
I20260926 19:58:31.927318 12645 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:58:31.933777 12645 webserver.cc:533] Webserver started at http://127.11.233.194:44413/ using document root <none> and password file <none>
I20260926 19:58:31.934723 12645 fs_manager.cc:362] Metadata directory not provided
I20260926 19:58:31.934978 12645 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:58:31.935449 12645 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:58:31.939944 12645 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/data/instance:
uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
format_stamp: "Formatted at 2026-09-26 19:58:31 on dist-test-slave-jcdj"
I20260926 19:58:31.941044 12645 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/instance:
uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
format_stamp: "Formatted at 2026-09-26 19:58:31 on dist-test-slave-jcdj"
I20260926 19:58:31.948402 12645 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260926 19:58:31.953897 12693 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:31.955159 12645 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260926 19:58:31.955516 12645 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal
uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
format_stamp: "Formatted at 2026-09-26 19:58:31 on dist-test-slave-jcdj"
I20260926 19:58:31.955863 12645 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:58:32.007124 12645 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:58:32.008661 12645 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:58:32.009156 12645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:58:32.012019 12645 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 19:58:32.016258 12645 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:58:32.016515 12645 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:32.016806 12645 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 19:58:32.017009 12645 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:32.133982 12687 raft_consensus.cc:493] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:32.134626 12687 raft_consensus.cc:515] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:32.136466 12687 leader_election.cc:290] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers db2490adb6e84090b780f52470e580ab (127.11.233.253:37913), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:32.137435 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:58:32.138156 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:32.139923 12228 leader_election.cc:304] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db9d871438324598b27dd67b31c017d9; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab
I20260926 19:58:32.140550 12687 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 19:58:32.161548 12645 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.194:37461
I20260926 19:58:32.161666 12805 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.194:37461 every 8 connection(s)
I20260926 19:58:32.164144 12645 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/data/info.pb
I20260926 19:58:32.166997 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 12645
I20260926 19:58:32.167770 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/instance
I20260926 19:58:32.193323 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.233.195:0
--local_ip_for_outbound_sockets=127.11.233.195
--webserver_interface=127.11.233.195
--webserver_port=0
--tserver_master_addrs=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--builtin_ntp_servers=127.11.233.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 19:58:32.229091 12808 heartbeater.cc:344] Connected to a master server at 127.11.233.254:33143
I20260926 19:58:32.230028 12808 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:32.231441 12808 heartbeater.cc:507] Master 127.11.233.254:33143 requested a full tablet report, sending...
I20260926 19:58:32.231591 12807 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
I20260926 19:58:32.231894 12807 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:32.232786 12807 heartbeater.cc:507] Master 127.11.233.253:37913 requested a full tablet report, sending...
I20260926 19:58:32.234740 12673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:32.235532 12316 ts_manager.cc:194] Registered new tserver with Master: 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:32.235742 12673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:32.236531 12241 ts_manager.cc:194] Registered new tserver with Master: 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:32.240149 12806 heartbeater.cc:344] Connected to a master server at 127.11.233.252:44243
I20260926 19:58:32.240514 12806 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:32.241295 12806 heartbeater.cc:507] Master 127.11.233.252:44243 requested a full tablet report, sending...
I20260926 19:58:32.243481 12396 ts_manager.cc:194] Registered new tserver with Master: 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:32.245689 12396 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.11.233.194:33049
W20260926 19:58:32.541442 12816 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:58:32.542114 12816 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:58:32.542686 12816 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260926 19:58:32.552412 12824 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:32.552901 12824 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:32.554759 12824 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:32.555540 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:32.555943 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:32.557655 12303 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:58:32.558429 12824 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260926 19:58:32.578430 12816 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:58:32.579348 12816 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.233.195
I20260926 19:58:32.644907 12816 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.233.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.233.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.233.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:58:32.646603 12816 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:58:32.648767 12816 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:58:32.663803 12826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:32.748706 12806 heartbeater.cc:499] Master 127.11.233.252:44243 was elected leader, sending a full tablet report...
I20260926 19:58:33.144730 12832 raft_consensus.cc:493] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:33.146131 12832 raft_consensus.cc:515] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:33.148926 12833 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:33.149382 12833 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:33.150668 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:58:33.151238 12326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db9d871438324598b27dd67b31c017d9 in term 4.
I20260926 19:58:33.151369 12833 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:33.153813 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:33.154526 12251 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db2490adb6e84090b780f52470e580ab in term 4.
I20260926 19:58:33.158149 12227 leader_election.cc:304] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db2490adb6e84090b780f52470e580ab, db9d871438324598b27dd67b31c017d9; no voters:
I20260926 19:58:33.159833 12301 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db2490adb6e84090b780f52470e580ab, db9d871438324598b27dd67b31c017d9; no voters:
I20260926 19:58:33.157159 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:33.168303 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:33.169710 12833 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260926 19:58:33.170107 12833 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:33.170629 12833 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 4 FOLLOWER]: Advancing to term 5
I20260926 19:58:33.170568 12832 leader_election.cc:290] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers db2490adb6e84090b780f52470e580ab (127.11.233.253:37913), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:33.171052 12832 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260926 19:58:33.172201 12832 raft_consensus.cc:493] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:33.173004 12832 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 4 FOLLOWER]: Advancing to term 5
I20260926 19:58:33.181777 12832 raft_consensus.cc:515] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:33.185410 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab"
I20260926 19:58:33.187312 12833 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:33.187973 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6"
I20260926 19:58:33.189896 12832 leader_election.cc:290] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 5 election: Requested vote from peers db2490adb6e84090b780f52470e580ab (127.11.233.253:37913), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:33.196033 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9"
I20260926 19:58:33.196841 12251 raft_consensus.cc:2393] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate db2490adb6e84090b780f52470e580ab in current term 5: Already voted for candidate db9d871438324598b27dd67b31c017d9 in this term.
I20260926 19:58:33.197203 12326 raft_consensus.cc:2393] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate db9d871438324598b27dd67b31c017d9 in current term 5: Already voted for candidate db2490adb6e84090b780f52470e580ab in this term.
I20260926 19:58:33.199755 12227 leader_election.cc:304] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db9d871438324598b27dd67b31c017d9; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab
I20260926 19:58:33.199926 12833 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 5 election: Requested vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:33.203289 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6"
I20260926 19:58:33.206758 12303 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:58:33.207712 12833 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260926 19:58:33.208549 12251 raft_consensus.cc:1240] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 5 FOLLOWER]: Rejecting Update request from peer 8a69f5870f8d461aa5e61f6999bb05c6 for earlier term 4. Current term is 5. Ops: []
I20260926 19:58:33.210893 12835 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260926 19:58:33.211618 12832 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260926 19:58:33.222226 12836 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 4 LEADER]: Stepping down as leader of term 4
I20260926 19:58:33.222802 12836 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a69f5870f8d461aa5e61f6999bb05c6, State: Running, Role: LEADER
I20260926 19:58:33.224079 12836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:33.227286 12836 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 4 FOLLOWER]: Advancing to term 5
I20260926 19:58:33.718439 12836 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:33.719065 12836 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:33.723917 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:33.724753 12251 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 5.
I20260926 19:58:33.726086 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:58:33.726851 12326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 5.
I20260926 19:58:33.730170 12382 leader_election.cc:304] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab; no voters:
I20260926 19:58:33.732478 12836 leader_election.cc:290] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), db2490adb6e84090b780f52470e580ab (127.11.233.253:37913)
I20260926 19:58:33.733198 12836 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260926 19:58:33.733696 12836 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:33.734174 12836 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 5 FOLLOWER]: Advancing to term 6
I20260926 19:58:33.765257 12836 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:33.780460 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9"
I20260926 19:58:33.781199 12251 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 5 FOLLOWER]: Advancing to term 6
I20260926 19:58:33.794651 12251 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 6.
I20260926 19:58:33.798208 12326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab"
I20260926 19:58:33.798950 12326 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 5 FOLLOWER]: Advancing to term 6
I20260926 19:58:33.830605 12836 leader_election.cc:290] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 6 election: Requested vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), db2490adb6e84090b780f52470e580ab (127.11.233.253:37913)
I20260926 19:58:33.834062 12326 raft_consensus.cc:2468] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a69f5870f8d461aa5e61f6999bb05c6 in term 6.
I20260926 19:58:33.836045 12381 leader_election.cc:304] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9; no voters:
I20260926 19:58:33.837292 12836 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 6 FOLLOWER]: Leader election won for term 6
I20260926 19:58:33.838270 12836 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [term 6 LEADER]: Becoming Leader. State: Replica: 8a69f5870f8d461aa5e61f6999bb05c6, State: Running, Role: LEADER
I20260926 19:58:33.839713 12836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:33.853124 12837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:33.854116 12837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:33.862639 12839 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 19:58:33.873567 12839 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 19:58:33.878999 12839 catalog_manager.cc:1308] Loaded cluster ID: 74efea6f15be4c5bb4893708cb04c269
I20260926 19:58:33.879401 12839 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:58:33.885579 12839 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:58:33.891021 12839 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Loaded TSK: 0
I20260926 19:58:33.892325 12839 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:58:34.356403 12251 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 6 FOLLOWER]: Refusing update from remote peer 8a69f5870f8d461aa5e61f6999bb05c6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260926 19:58:34.358152 12837 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260926 19:58:34.380394 12326 raft_consensus.cc:1275] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Refusing update from remote peer 8a69f5870f8d461aa5e61f6999bb05c6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260926 19:58:34.382694 12837 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260926 19:58:34.385780 12841 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:34.386458 12841 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:34.408792 12841 sys_catalog.cc:455] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:34.409585 12841 sys_catalog.cc:458] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:34.414529 12843 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:34.415463 12843 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:34.415110 12837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:34.415903 12837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
W20260926 19:58:34.067476 12825 debug-util.cc:398] Leaking SignalData structure 0x7b0800037180 after lost signal to thread 12816
W20260926 19:58:34.419637 12825 kernel_stack_watchdog.cc:198] Thread 12816 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 399ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260926 19:58:32.664983 12827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:58:34.421208 12816 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.755s user 0.002s sys 0.000s
W20260926 19:58:34.421586 12816 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.756s user 0.002s sys 0.000s
I20260926 19:58:34.426520 12837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:34.427255 12837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
W20260926 19:58:34.428033 12828 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1760 milliseconds
I20260926 19:58:34.429018 12843 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:34.429802 12843 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:34.430145 12816 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260926 19:58:34.430553 12829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:34.431751 12816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:58:34.433910 12816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:58:34.435357 12816 hybrid_clock.cc:648] HybridClock initialized: now 1790452714435317 us; error 38 us; skew 500 ppm
I20260926 19:58:34.436204 12816 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:58:34.442337 12816 webserver.cc:533] Webserver started at http://127.11.233.195:40905/ using document root <none> and password file <none>
I20260926 19:58:34.443259 12816 fs_manager.cc:362] Metadata directory not provided
I20260926 19:58:34.443485 12816 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:58:34.443967 12816 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:58:34.448276 12816 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/data/instance:
uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
format_stamp: "Formatted at 2026-09-26 19:58:34 on dist-test-slave-jcdj"
I20260926 19:58:34.449410 12816 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/instance:
uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
format_stamp: "Formatted at 2026-09-26 19:58:34 on dist-test-slave-jcdj"
I20260926 19:58:34.456945 12816 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260926 19:58:34.463218 12856 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:34.464509 12816 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
I20260926 19:58:34.464922 12816 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal
uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
format_stamp: "Formatted at 2026-09-26 19:58:34 on dist-test-slave-jcdj"
I20260926 19:58:34.465263 12816 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:58:34.539194 12816 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:58:34.540705 12816 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:58:34.541163 12816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:58:34.543893 12816 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 19:58:34.548035 12816 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:58:34.548233 12816 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:34.548604 12816 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 19:58:34.548786 12816 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:34.718977 12816 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.195:44901
I20260926 19:58:34.719087 12968 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.195:44901 every 8 connection(s)
I20260926 19:58:34.721630 12816 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/data/info.pb
I20260926 19:58:34.725582 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 12816
I20260926 19:58:34.726071 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/instance
I20260926 19:58:34.778703 12969 heartbeater.cc:344] Connected to a master server at 127.11.233.252:44243
I20260926 19:58:34.779241 12969 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:34.779968 12970 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
I20260926 19:58:34.780292 12970 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:34.780639 12969 heartbeater.cc:507] Master 127.11.233.252:44243 requested a full tablet report, sending...
I20260926 19:58:34.781774 12971 heartbeater.cc:344] Connected to a master server at 127.11.233.254:33143
I20260926 19:58:34.782079 12970 heartbeater.cc:507] Master 127.11.233.253:37913 requested a full tablet report, sending...
I20260926 19:58:34.782081 12971 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:34.783154 12971 heartbeater.cc:507] Master 127.11.233.254:33143 requested a full tablet report, sending...
I20260926 19:58:34.784260 12806 heartbeater.cc:499] Master 127.11.233.252:44243 was elected leader, sending a full tablet report...
I20260926 19:58:34.784777 12396 ts_manager.cc:194] Registered new tserver with Master: 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:34.785851 12241 ts_manager.cc:194] Registered new tserver with Master: 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:34.786294 12316 ts_manager.cc:194] Registered new tserver with Master: 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:34.788609 12396 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.11.233.195:37269
I20260926 19:58:34.807312 12199 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260926 19:58:34.811508 12640 heartbeater.cc:499] Master 127.11.233.252:44243 was elected leader, sending a full tablet report...
I20260926 19:58:34.878072 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_163ace87739e4fc19e5ee50ff2bbef14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:34.878098 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_74bd733f91cb416fb5c8bc5178943b88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:34.880277 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_3bd8b5e31113456d9a83056eb52aa203"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:34.881897 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_163ace87739e4fc19e5ee50ff2bbef14 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:34.881925 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74bd733f91cb416fb5c8bc5178943b88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:34.882747 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bd8b5e31113456d9a83056eb52aa203 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:34.885950 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ac98016959a64b5f8718a43ba4bf5bf7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:34.888697 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac98016959a64b5f8718a43ba4bf5bf7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:34.913939 12199 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu with pid 12278
W20260926 19:58:34.949934 12382 connection.cc:573] client connection to 127.11.233.253:37913 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:58:34.950384 12382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:58:34.966449 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_74bd733f91cb416fb5c8bc5178943b88" } new_table_name: "default.Table_88f5f317358c4148a7264865cc1c1ccf" modify_external_catalogs: true
W20260926 19:58:34.970141 12382 consensus_peers.cc:597] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 -> Peer db2490adb6e84090b780f52470e580ab (127.11.233.253:37913): Couldn't send request to peer db2490adb6e84090b780f52470e580ab. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:34.970321 12394 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_ac98016959a64b5f8718a43ba4bf5bf7" } new_table_name: "default.Table_04ed2c317fcf4273ab63c0dc3fa8febc" modify_external_catalogs: true
I20260926 19:58:34.971818 12395 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3bd8b5e31113456d9a83056eb52aa203" } modify_external_catalogs: true
I20260926 19:58:34.972595 12395 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3bd8b5e31113456d9a83056eb52aa203" } modify_external_catalogs: true
W20260926 19:58:34.990028 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdcecfe46c3447e493f1ab8b5eff3c62 has no consensus state
I20260926 19:58:34.998426 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_163ace87739e4fc19e5ee50ff2bbef14" } modify_external_catalogs: true
I20260926 19:58:34.999212 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_163ace87739e4fc19e5ee50ff2bbef14" } modify_external_catalogs: true
W20260926 19:58:35.000986 12394 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6166f0b9fe24ceb8e83b9910651d156 has no consensus state
I20260926 19:58:35.016367 12395 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet ce3999b11dab439b828357d7e49cc803
W20260926 19:58:35.023684 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_88f5f317358c4148a7264865cc1c1ccf [id=01bf90b95d6c402a9dbdf3e5b3f760d6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bdcecfe46c3447e493f1ab8b5eff3c62 consensus state has no leader
I20260926 19:58:35.043115 12392 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 0c9b8b54d9dc49d5bacfef35b9e07a65
I20260926 19:58:35.051754 12740 tablet_service.cc:1511] Processing CreateTablet for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (DEFAULT_TABLE table=default.Table_88f5f317358c4148a7264865cc1c1ccf [id=01bf90b95d6c402a9dbdf3e5b3f760d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.051326 12741 tablet_service.cc:1511] Processing CreateTablet for tablet d6166f0b9fe24ceb8e83b9910651d156 (DEFAULT_TABLE table=default.Table_04ed2c317fcf4273ab63c0dc3fa8febc [id=418957ebf6b5490abef44b6064f7154a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.053647 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6166f0b9fe24ceb8e83b9910651d156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.054291 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdcecfe46c3447e493f1ab8b5eff3c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.056535 12568 tablet_service.cc:1511] Processing CreateTablet for tablet d6166f0b9fe24ceb8e83b9910651d156 (DEFAULT_TABLE table=default.Table_04ed2c317fcf4273ab63c0dc3fa8febc [id=418957ebf6b5490abef44b6064f7154a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.059556 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6166f0b9fe24ceb8e83b9910651d156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.075507 12904 tablet_service.cc:1511] Processing CreateTablet for tablet d6166f0b9fe24ceb8e83b9910651d156 (DEFAULT_TABLE table=default.Table_04ed2c317fcf4273ab63c0dc3fa8febc [id=418957ebf6b5490abef44b6064f7154a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.079277 12903 tablet_service.cc:1511] Processing CreateTablet for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (DEFAULT_TABLE table=default.Table_88f5f317358c4148a7264865cc1c1ccf [id=01bf90b95d6c402a9dbdf3e5b3f760d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.081004 12567 tablet_service.cc:1511] Processing CreateTablet for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (DEFAULT_TABLE table=default.Table_88f5f317358c4148a7264865cc1c1ccf [id=01bf90b95d6c402a9dbdf3e5b3f760d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.082832 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdcecfe46c3447e493f1ab8b5eff3c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.085939 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdcecfe46c3447e493f1ab8b5eff3c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.090276 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6166f0b9fe24ceb8e83b9910651d156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.098277 13012 tablet_bootstrap.cc:492] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:35.142474 13012 tablet_bootstrap.cc:654] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.148216 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_5690ea27eebd457ab30ece80ffe90f09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:35.150831 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5690ea27eebd457ab30ece80ffe90f09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.153899 13012 log.cc:826] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Log is configured to *not* fsync() on all Append() calls
I20260926 19:58:35.157231 13013 tablet_bootstrap.cc:492] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:35.159783 13014 tablet_bootstrap.cc:492] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:35.165228 13013 tablet_bootstrap.cc:654] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.167646 13013 log.cc:826] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Log is configured to *not* fsync() on all Append() calls
I20260926 19:58:35.169375 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_eb7c0d6805e24e2eb87c5b2b75d04edc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:35.171708 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb7c0d6805e24e2eb87c5b2b75d04edc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.178639 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_73148470e8e44a98880dc32126902294"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:35.179978 13014 tablet_bootstrap.cc:654] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.180267 13012 tablet_bootstrap.cc:492] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
W20260926 19:58:35.180924 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73148470e8e44a98880dc32126902294 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.181895 13012 ts_tablet_manager.cc:1403] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.085s user 0.016s sys 0.004s
I20260926 19:58:35.182410 13014 log.cc:826] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Log is configured to *not* fsync() on all Append() calls
I20260926 19:58:35.199011 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_1c090c26a87e499baeb08c7be0703ca2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:35.201287 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c090c26a87e499baeb08c7be0703ca2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.209206 12393 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_88f5f317358c4148a7264865cc1c1ccf" } new_table_name: "default.Table_ee0eba1e9d7548799bca25f3815c5ad3" modify_external_catalogs: true
I20260926 19:58:35.214414 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_04ed2c317fcf4273ab63c0dc3fa8febc" } new_table_name: "default.Table_b1e07cc33f1143909a84c80f7d9ad5c5" modify_external_catalogs: true
I20260926 19:58:35.254525 13012 raft_consensus.cc:359] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.255458 13012 raft_consensus.cc:385] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.255760 13012 raft_consensus.cc:740] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.256726 13012 consensus_queue.cc:260] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.266170 13012 ts_tablet_manager.cc:1434] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.084s user 0.047s sys 0.003s
I20260926 19:58:35.266472 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5690ea27eebd457ab30ece80ffe90f09" } modify_external_catalogs: true
I20260926 19:58:35.266889 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1c090c26a87e499baeb08c7be0703ca2" } modify_external_catalogs: true
I20260926 19:58:35.267112 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5690ea27eebd457ab30ece80ffe90f09" } modify_external_catalogs: true
I20260926 19:58:35.267421 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1c090c26a87e499baeb08c7be0703ca2" } modify_external_catalogs: true
I20260926 19:58:35.274191 13014 tablet_bootstrap.cc:492] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:35.274765 13014 ts_tablet_manager.cc:1403] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.116s user 0.030s sys 0.037s
I20260926 19:58:35.281349 13012 tablet_bootstrap.cc:492] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:35.286502 12807 heartbeater.cc:646] Failed to heartbeat to 127.11.233.253:37913 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
I20260926 19:58:35.293778 12969 heartbeater.cc:499] Master 127.11.233.252:44243 was elected leader, sending a full tablet report...
W20260926 19:58:35.312376 12970 heartbeater.cc:646] Failed to heartbeat to 127.11.233.253:37913 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
I20260926 19:58:35.313866 13013 tablet_bootstrap.cc:492] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:35.314456 13013 ts_tablet_manager.cc:1403] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.158s user 0.057s sys 0.067s
I20260926 19:58:35.300837 13012 tablet_bootstrap.cc:654] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.323614 13014 raft_consensus.cc:359] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.324510 13014 raft_consensus.cc:385] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.324826 13014 raft_consensus.cc:740] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.325908 13014 consensus_queue.cc:260] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.380754 13013 raft_consensus.cc:359] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.381862 13013 raft_consensus.cc:385] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.382217 13013 raft_consensus.cc:740] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:35.382409 12641 heartbeater.cc:646] Failed to heartbeat to 127.11.233.253:37913 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111)
I20260926 19:58:35.383328 13013 consensus_queue.cc:260] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.395417 12837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:35.396674 12837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:58:35.400023 13013 ts_tablet_manager.cc:1434] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.082s user 0.026s sys 0.026s
I20260926 19:58:35.401099 13013 tablet_bootstrap.cc:492] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:35.407325 13013 tablet_bootstrap.cc:654] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.419377 13013 tablet_bootstrap.cc:492] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:35.419937 13013 ts_tablet_manager.cc:1403] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260926 19:58:35.422569 13012 tablet_bootstrap.cc:492] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:35.423134 13012 ts_tablet_manager.cc:1403] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.142s user 0.013s sys 0.007s
I20260926 19:58:35.426997 13012 raft_consensus.cc:359] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.427939 13012 raft_consensus.cc:385] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.428316 13012 raft_consensus.cc:740] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.429160 13012 consensus_queue.cc:260] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:35.430286 12381 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.254:33143, user_credentials={real_user=slave}} blocked reactor thread for 45404.9us
I20260926 19:58:35.441673 13012 ts_tablet_manager.cc:1434] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.018s user 0.010s sys 0.000s
I20260926 19:58:35.442996 13013 raft_consensus.cc:359] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.443711 13013 raft_consensus.cc:385] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.444013 13013 raft_consensus.cc:740] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.444697 13013 consensus_queue.cc:260] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.446882 13013 ts_tablet_manager.cc:1434] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
I20260926 19:58:35.450433 12390 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_88f5f317358c4148a7264865cc1c1ccf [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 1 got 0
I20260926 19:58:35.452677 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_04ed2c317fcf4273ab63c0dc3fa8febc [id=418957ebf6b5490abef44b6064f7154a]). Expected version 1 got 0
I20260926 19:58:35.453315 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_88f5f317358c4148a7264865cc1c1ccf [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 1 got 0
W20260926 19:58:35.454552 12393 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdcecfe46c3447e493f1ab8b5eff3c62 consensus state has no leader
I20260926 19:58:35.455710 12388 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_04ed2c317fcf4273ab63c0dc3fa8febc [id=418957ebf6b5490abef44b6064f7154a]). Expected version 1 got 0
W20260926 19:58:35.457495 12813 tablet.cc:2433] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:58:35.457994 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6166f0b9fe24ceb8e83b9910651d156 consensus state has no leader
I20260926 19:58:35.459600 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 (DEFAULT_TABLE table=default.Table_1c090c26a87e499baeb08c7be0703ca2 [id=6187d80c67d54a229a0868f99c892896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.460907 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b229bf63055460dbb6b6df6f3c55887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.466572 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 (DEFAULT_TABLE table=default.Table_1c090c26a87e499baeb08c7be0703ca2 [id=6187d80c67d54a229a0868f99c892896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.468062 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b229bf63055460dbb6b6df6f3c55887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.471846 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 (DEFAULT_TABLE table=default.Table_1c090c26a87e499baeb08c7be0703ca2 [id=6187d80c67d54a229a0868f99c892896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.473276 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b229bf63055460dbb6b6df6f3c55887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.473488 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_0ac426c9cb7f43e79751ee814930be7f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:35.473565 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ab469173bd8d4ff197f4240a8a86fd71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:35.475320 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab469173bd8d4ff197f4240a8a86fd71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:35.475339 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ac426c9cb7f43e79751ee814930be7f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.480530 13014 ts_tablet_manager.cc:1434] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.205s user 0.071s sys 0.053s
I20260926 19:58:35.481570 13014 tablet_bootstrap.cc:492] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:35.487409 13014 tablet_bootstrap.cc:654] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.488821 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_b1e07cc33f1143909a84c80f7d9ad5c5 [id=418957ebf6b5490abef44b6064f7154a]). Expected version 2 got 0
I20260926 19:58:35.489432 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 2 got 0
I20260926 19:58:35.500447 13026 raft_consensus.cc:493] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:35.501255 13026 raft_consensus.cc:515] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.504748 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_b1e07cc33f1143909a84c80f7d9ad5c5 [id=418957ebf6b5490abef44b6064f7154a]). Expected version 2 got 0
W20260926 19:58:35.513276 12976 tablet.cc:2433] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:58:35.515594 13026 leader_election.cc:290] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:35.520711 12391 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_b1e07cc33f1143909a84c80f7d9ad5c5 [id=418957ebf6b5490abef44b6064f7154a]). Expected version 2 got 0
I20260926 19:58:35.528766 12396 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 39984d7be4064f71a92f5ab4c46b1834
I20260926 19:58:35.530123 12393 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_b1e07cc33f1143909a84c80f7d9ad5c5 [id=418957ebf6b5490abef44b6064f7154a]). Expected version 2 got 0
I20260926 19:58:35.539165 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 5b229bf63055460dbb6b6df6f3c55887
I20260926 19:58:35.549209 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:33594
W20260926 19:58:35.532994 12649 tablet.cc:2433] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:58:35.550810 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_7ff1028532b84f1786552f6dcd887986"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:35.554551 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:35.556705 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ff1028532b84f1786552f6dcd887986 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.562713 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:35.583874 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6166f0b9fe24ceb8e83b9910651d156" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:35.584833 12761 raft_consensus.cc:2468] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:35.586527 12519 leader_election.cc:304] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:35.587383 13026 raft_consensus.cc:2804] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:35.587845 13026 raft_consensus.cc:493] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:35.588277 13026 raft_consensus.cc:3060] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:35.598248 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 5b229bf63055460dbb6b6df6f3c55887 because the tablet was not found. No further retry: Not found: Tablet not found: 5b229bf63055460dbb6b6df6f3c55887
I20260926 19:58:35.602381 13018 raft_consensus.cc:493] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:35.602887 13018 raft_consensus.cc:515] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:35.604885 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 5b229bf63055460dbb6b6df6f3c55887 because the tablet was not found. No further retry: Not found: Tablet not found: 5b229bf63055460dbb6b6df6f3c55887
I20260926 19:58:35.607285 13018 leader_election.cc:290] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:35.625077 13036 raft_consensus.cc:493] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:35.626971 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9ce01afc99f748de89f811d40343895d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:35.629011 13026 raft_consensus.cc:515] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:35.630115 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ce01afc99f748de89f811d40343895d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.629803 13036 raft_consensus.cc:515] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.632205 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6166f0b9fe24ceb8e83b9910651d156" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:35.632767 12761 raft_consensus.cc:3060] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:35.630888 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 5b229bf63055460dbb6b6df6f3c55887 because the tablet was not found. No further retry: Not found: Tablet not found: 5b229bf63055460dbb6b6df6f3c55887
I20260926 19:58:35.635926 13026 leader_election.cc:290] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:35.638137 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdcecfe46c3447e493f1ab8b5eff3c62" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:35.638847 12760 raft_consensus.cc:2468] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:35.639714 12761 raft_consensus.cc:2468] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:35.640327 12519 leader_election.cc:304] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:35.640305 13036 leader_election.cc:290] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:35.642190 12519 leader_election.cc:304] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:35.643326 13013 tablet_bootstrap.cc:492] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:35.643859 13036 raft_consensus.cc:2804] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:35.644196 13036 raft_consensus.cc:493] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:35.644546 13036 raft_consensus.cc:3060] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:35.645359 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6166f0b9fe24ceb8e83b9910651d156" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:35.646229 12924 raft_consensus.cc:2468] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:35.651163 13013 tablet_bootstrap.cc:654] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.652341 13014 tablet_bootstrap.cc:492] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:35.652797 13014 ts_tablet_manager.cc:1403] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.171s user 0.047s sys 0.032s
I20260926 19:58:35.652913 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6166f0b9fe24ceb8e83b9910651d156" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:35.653429 12924 raft_consensus.cc:3060] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:35.656162 13014 raft_consensus.cc:359] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.658437 13026 raft_consensus.cc:2804] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:35.661247 13012 tablet_bootstrap.cc:492] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:35.665194 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 4e706596959e400398bf2213653f06c9 (DEFAULT_TABLE table=default.Table_eb7c0d6805e24e2eb87c5b2b75d04edc [id=e2cf63902d2c4d28815fca606eef0274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.667582 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e706596959e400398bf2213653f06c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.667908 13014 raft_consensus.cc:385] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.660434 12924 raft_consensus.cc:2468] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:35.662289 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdcecfe46c3447e493f1ab8b5eff3c62" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:35.668188 13012 tablet_bootstrap.cc:654] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.668313 12741 tablet_service.cc:1511] Processing CreateTablet for tablet a288f85f189f4b7baca505928471bb68 (DEFAULT_TABLE table=default.Table_73148470e8e44a98880dc32126902294 [id=4b885f1ea09a42caa59023689fcc46f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.671097 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a288f85f189f4b7baca505928471bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.672655 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 4e706596959e400398bf2213653f06c9 (DEFAULT_TABLE table=default.Table_eb7c0d6805e24e2eb87c5b2b75d04edc [id=e2cf63902d2c4d28815fca606eef0274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.672940 12567 tablet_service.cc:1511] Processing CreateTablet for tablet a288f85f189f4b7baca505928471bb68 (DEFAULT_TABLE table=default.Table_73148470e8e44a98880dc32126902294 [id=4b885f1ea09a42caa59023689fcc46f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.673771 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e706596959e400398bf2213653f06c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.674351 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a288f85f189f4b7baca505928471bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.674393 12904 tablet_service.cc:1511] Processing CreateTablet for tablet a288f85f189f4b7baca505928471bb68 (DEFAULT_TABLE table=default.Table_73148470e8e44a98880dc32126902294 [id=4b885f1ea09a42caa59023689fcc46f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.675428 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a288f85f189f4b7baca505928471bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.680064 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_eb7c0d6805e24e2eb87c5b2b75d04edc" } new_table_name: "default.Table_0ce88298a7c04aebabf03fcecc58bd5a" modify_external_catalogs: true
I20260926 19:58:35.683947 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_0ac426c9cb7f43e79751ee814930be7f" } new_table_name: "default.Table_9729e85fef9b43b7bc6f5a034c04e20e" modify_external_catalogs: true
I20260926 19:58:35.668259 13014 raft_consensus.cc:740] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.690637 13014 consensus_queue.cc:260] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.692283 12923 raft_consensus.cc:2468] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:35.701809 13014 ts_tablet_manager.cc:1434] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.049s user 0.004s sys 0.003s
I20260926 19:58:35.702759 13014 tablet_bootstrap.cc:492] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:35.716100 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 4e706596959e400398bf2213653f06c9 (DEFAULT_TABLE table=default.Table_eb7c0d6805e24e2eb87c5b2b75d04edc [id=e2cf63902d2c4d28815fca606eef0274]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.718245 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e706596959e400398bf2213653f06c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.723688 13026 raft_consensus.cc:697] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:35.726203 13013 tablet_bootstrap.cc:492] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:35.726630 13013 ts_tablet_manager.cc:1403] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.084s user 0.005s sys 0.012s
I20260926 19:58:35.729395 13013 raft_consensus.cc:359] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.729786 13036 raft_consensus.cc:515] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.730108 13013 raft_consensus.cc:385] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.729931 13026 consensus_queue.cc:237] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.730525 13013 raft_consensus.cc:740] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.731212 13013 consensus_queue.cc:260] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.733384 13014 tablet_bootstrap.cc:654] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.734036 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_7ff1028532b84f1786552f6dcd887986" } modify_external_catalogs: true
I20260926 19:58:35.734637 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_7ff1028532b84f1786552f6dcd887986" } modify_external_catalogs: true
I20260926 19:58:35.739013 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdcecfe46c3447e493f1ab8b5eff3c62" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:35.739563 12761 raft_consensus.cc:3060] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:35.740491 13036 leader_election.cc:290] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:35.743307 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdcecfe46c3447e493f1ab8b5eff3c62" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:35.743912 12923 raft_consensus.cc:3060] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:35.751731 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_ab469173bd8d4ff197f4240a8a86fd71" } modify_external_catalogs: true
I20260926 19:58:35.752267 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_ab469173bd8d4ff197f4240a8a86fd71" } modify_external_catalogs: true
I20260926 19:58:35.752527 12923 raft_consensus.cc:2468] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:35.753865 12521 leader_election.cc:304] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:35.754981 13039 raft_consensus.cc:2804] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:35.755498 13039 raft_consensus.cc:697] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:35.776494 13039 consensus_queue.cc:237] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.779327 12353 master_runner.cc:187] Error getting master registration for 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111),
I20260926 19:58:35.779860 12353 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:58:35.781044 13013 ts_tablet_manager.cc:1434] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.054s user 0.006s sys 0.000s
I20260926 19:58:35.791710 13013 tablet_bootstrap.cc:492] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:35.799839 12761 raft_consensus.cc:2468] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:35.806908 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdcecfe46c3447e493f1ab8b5eff3c62" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:35.811041 13014 tablet_bootstrap.cc:492] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:35.811595 13014 ts_tablet_manager.cc:1403] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.109s user 0.012s sys 0.004s
I20260926 19:58:35.816742 13013 tablet_bootstrap.cc:654] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.821859 13014 raft_consensus.cc:359] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.822548 13014 raft_consensus.cc:385] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.822829 13014 raft_consensus.cc:740] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.823606 13014 consensus_queue.cc:260] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.823786 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_26612903c7e7487bb95439ebde7b204e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:35.825486 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_26612903c7e7487bb95439ebde7b204e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.830433 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_68d42bf187a146958c5daf317378dafd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:35.831451 13014 ts_tablet_manager.cc:1434] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
W20260926 19:58:35.832155 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68d42bf187a146958c5daf317378dafd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:35.840303 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e706596959e400398bf2213653f06c9 consensus state has no leader
I20260926 19:58:35.841483 13014 tablet_bootstrap.cc:492] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:35.847667 13014 tablet_bootstrap.cc:654] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.851806 13013 tablet_bootstrap.cc:492] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:35.852219 13013 ts_tablet_manager.cc:1403] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.008s
I20260926 19:58:35.855743 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdcecfe46c3447e493f1ab8b5eff3c62" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:35.856491 12923 raft_consensus.cc:2393] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:35.857791 13012 tablet_bootstrap.cc:492] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:35.858278 13012 ts_tablet_manager.cc:1403] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.197s user 0.014s sys 0.004s
I20260926 19:58:35.858029 12696 leader_election.cc:304] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:35.859393 13018 raft_consensus.cc:2749] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:35.855232 13013 raft_consensus.cc:359] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.860526 13013 raft_consensus.cc:385] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.860812 13013 raft_consensus.cc:740] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.861234 13012 raft_consensus.cc:359] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.861483 13013 consensus_queue.cc:260] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.862329 13012 raft_consensus.cc:385] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.865603 13012 raft_consensus.cc:740] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.874284 13013 ts_tablet_manager.cc:1434] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.022s user 0.005s sys 0.004s
I20260926 19:58:35.880396 13023 raft_consensus.cc:493] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:35.881047 13023 raft_consensus.cc:515] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:35.885026 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8538b46e783f4ffb9a31c12939f954af has no consensus state
I20260926 19:58:35.890491 13023 leader_election.cc:290] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:35.889956 13012 consensus_queue.cc:260] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.895398 13013 tablet_bootstrap.cc:492] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:35.914453 13013 tablet_bootstrap.cc:654] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:35.915105 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a6745a20e46f46c8bda622354e9d558b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:35.918686 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b229bf63055460dbb6b6df6f3c55887" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:35.918983 13014 tablet_bootstrap.cc:492] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:35.919411 13014 ts_tablet_manager.cc:1403] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.078s user 0.015s sys 0.013s
I20260926 19:58:35.919310 12595 raft_consensus.cc:2468] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:35.921190 12858 leader_election.cc:304] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:35.922163 13023 raft_consensus.cc:2804] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:35.922482 13023 raft_consensus.cc:493] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:35.922794 13023 raft_consensus.cc:3060] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:35.925606 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6745a20e46f46c8bda622354e9d558b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:35.927175 13014 raft_consensus.cc:359] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.927855 13014 raft_consensus.cc:385] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.928134 13014 raft_consensus.cc:740] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.929883 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:35.933322 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b229bf63055460dbb6b6df6f3c55887" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:35.933921 12761 raft_consensus.cc:2468] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
W20260926 19:58:35.934463 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 5b229bf63055460dbb6b6df6f3c55887 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b229bf63055460dbb6b6df6f3c55887 already in progress: creating tablet
I20260926 19:58:35.936908 13023 raft_consensus.cc:515] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.928831 13014 consensus_queue.cc:260] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.942478 13013 tablet_bootstrap.cc:492] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:35.943358 13013 ts_tablet_manager.cc:1403] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.002s
I20260926 19:58:35.946069 13013 raft_consensus.cc:359] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.946725 13013 raft_consensus.cc:385] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:35.946993 13013 raft_consensus.cc:740] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:35.954206 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 2 got 0
I20260926 19:58:35.954797 12568 tablet_service.cc:1511] Processing CreateTablet for tablet de292b22227a440fb2dc9e9a11f208c2 (DEFAULT_TABLE table=default.Table_ab469173bd8d4ff197f4240a8a86fd71 [id=27fa522794c84e1f96b15a4bb6b5570b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.955837 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de292b22227a440fb2dc9e9a11f208c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.956826 12390 catalog_manager.cc:5838] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:35.958541 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_b1e07cc33f1143909a84c80f7d9ad5c5 [id=418957ebf6b5490abef44b6064f7154a]). Expected version 2 got 0
I20260926 19:58:35.959172 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b229bf63055460dbb6b6df6f3c55887" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:35.959692 12595 raft_consensus.cc:3060] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:35.962329 12904 tablet_service.cc:1511] Processing CreateTablet for tablet de292b22227a440fb2dc9e9a11f208c2 (DEFAULT_TABLE table=default.Table_ab469173bd8d4ff197f4240a8a86fd71 [id=27fa522794c84e1f96b15a4bb6b5570b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.963464 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de292b22227a440fb2dc9e9a11f208c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.966526 12595 raft_consensus.cc:2468] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:35.967664 12858 leader_election.cc:304] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:35.967725 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 8538b46e783f4ffb9a31c12939f954af (DEFAULT_TABLE table=default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.970865 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 8538b46e783f4ffb9a31c12939f954af (DEFAULT_TABLE table=default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.972031 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8538b46e783f4ffb9a31c12939f954af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.947647 13013 consensus_queue.cc:260] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:35.973161 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b229bf63055460dbb6b6df6f3c55887" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:35.973706 12761 raft_consensus.cc:3060] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:35.974395 13013 ts_tablet_manager.cc:1434] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.031s user 0.004s sys 0.002s
I20260926 19:58:35.976261 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 8538b46e783f4ffb9a31c12939f954af (DEFAULT_TABLE table=default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:35.969431 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8538b46e783f4ffb9a31c12939f954af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.977260 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8538b46e783f4ffb9a31c12939f954af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.985639 12761 raft_consensus.cc:2468] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:35.987504 13023 leader_election.cc:290] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:35.997083 13013 tablet_bootstrap.cc:492] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:35.997792 13023 raft_consensus.cc:2804] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:35.998387 13023 raft_consensus.cc:697] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:35.999608 12740 tablet_service.cc:1511] Processing CreateTablet for tablet de292b22227a440fb2dc9e9a11f208c2 (DEFAULT_TABLE table=default.Table_ab469173bd8d4ff197f4240a8a86fd71 [id=27fa522794c84e1f96b15a4bb6b5570b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.004031 13013 tablet_bootstrap.cc:654] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.006027 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de292b22227a440fb2dc9e9a11f208c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:35.999650 13023 consensus_queue.cc:237] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.022576 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 2 got 0
I20260926 19:58:36.028970 13013 tablet_bootstrap.cc:492] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:36.029398 13013 ts_tablet_manager.cc:1403] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.011s
I20260926 19:58:35.912763 13012 ts_tablet_manager.cc:1434] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.054s user 0.007s sys 0.000s
I20260926 19:58:36.032028 13013 raft_consensus.cc:359] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.033123 13013 raft_consensus.cc:385] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.033412 13013 raft_consensus.cc:740] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.038542 13012 tablet_bootstrap.cc:492] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:36.038098 13013 consensus_queue.cc:260] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.041818 13013 ts_tablet_manager.cc:1434] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260926 19:58:36.046027 13013 tablet_bootstrap.cc:492] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
W20260926 19:58:36.052016 12380 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 4e706596959e400398bf2213653f06c9 (table default.Table_0ce88298a7c04aebabf03fcecc58bd5a [id=e2cf63902d2c4d28815fca606eef0274], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4e706596959e400398bf2213653f06c9 consensus state has no leader
W20260926 19:58:36.057318 12382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.253:37913, user_credentials={real_user=slave}} blocked reactor thread for 66256us
I20260926 19:58:36.058095 13013 tablet_bootstrap.cc:654] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.058887 13039 raft_consensus.cc:493] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:36.059307 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:36.059309 13039 raft_consensus.cc:515] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.060290 13031 tablet_replica.cc:333] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:36.061118 13031 raft_consensus.cc:2243] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:36.062330 13031 pending_rounds.cc:70] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Trying to abort 1 pending ops.
I20260926 19:58:36.062608 13031 pending_rounds.cc:77] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694324821860352 op_type: NO_OP noop_request { }
I20260926 19:58:36.063004 13031 raft_consensus.cc:2889] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:36.063344 13031 raft_consensus.cc:2272] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.063465 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a288f85f189f4b7baca505928471bb68" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:36.064044 12923 raft_consensus.cc:2468] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:36.065886 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a288f85f189f4b7baca505928471bb68" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:36.067099 13031 ts_tablet_manager.cc:1916] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.067353 12521 leader_election.cc:304] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:36.068085 13026 raft_consensus.cc:2804] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:36.068425 13026 raft_consensus.cc:493] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:36.068737 13026 raft_consensus.cc:3060] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:36.069118 12519 leader_election.cc:343] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:36.061276 13039 leader_election.cc:290] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:36.075466 13026 raft_consensus.cc:515] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.075706 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_55128b0718b349d18a8f378ecd6d0a24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:36.078135 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a288f85f189f4b7baca505928471bb68" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
W20260926 19:58:36.079499 12519 leader_election.cc:343] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:36.080994 13039 raft_consensus.cc:493] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:36.081041 13026 leader_election.cc:290] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:36.081481 13039 raft_consensus.cc:515] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:36.083972 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55128b0718b349d18a8f378ecd6d0a24 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:36.084937 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e706596959e400398bf2213653f06c9" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:36.086520 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a288f85f189f4b7baca505928471bb68" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:36.087033 12923 raft_consensus.cc:3060] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:36.090847 12519 leader_election.cc:343] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:36.090993 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:36.092139 13032 tablet_replica.cc:333] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:36.092705 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9ce01afc99f748de89f811d40343895d" } new_table_name: "default.Table_f6cb69a2397348c3a00214884a88d013" modify_external_catalogs: true
I20260926 19:58:36.093346 13032 raft_consensus.cc:2243] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:36.094110 13032 raft_consensus.cc:2272] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.094626 13039 leader_election.cc:290] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
W20260926 19:58:36.095667 12521 leader_election.cc:343] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Not found: Tablet not found: 4e706596959e400398bf2213653f06c9
I20260926 19:58:36.096092 12521 leader_election.cc:304] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:36.097236 13039 raft_consensus.cc:2749] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:36.097317 13032 ts_tablet_manager.cc:1916] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.098984 12923 raft_consensus.cc:2468] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:36.100203 12521 leader_election.cc:304] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:36.101195 13039 raft_consensus.cc:2804] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:36.103214 13039 raft_consensus.cc:697] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:36.104117 13039 consensus_queue.cc:237] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.111161 12393 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet be7355914de344cfb894db09e63522fb
I20260926 19:58:36.126899 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_0ce88298a7c04aebabf03fcecc58bd5a" } new_table_name: "default.Table_60d74638cb464ebd99d13cfed16bc764" modify_external_catalogs: true
I20260926 19:58:36.130340 13031 ts_tablet_manager.cc:1929] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:36.130949 13031 log.cc:1199] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/5b229bf63055460dbb6b6df6f3c55887
I20260926 19:58:36.135973 13013 tablet_bootstrap.cc:492] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:36.136493 13013 ts_tablet_manager.cc:1403] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.091s user 0.009s sys 0.016s
I20260926 19:58:36.138350 13014 ts_tablet_manager.cc:1434] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.219s user 0.036s sys 0.052s
I20260926 19:58:36.133862 13031 ts_tablet_manager.cc:1950] T 5b229bf63055460dbb6b6df6f3c55887 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:36.142184 13013 raft_consensus.cc:359] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.142215 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 5b229bf63055460dbb6b6df6f3c55887 (table default.Table_1c090c26a87e499baeb08c7be0703ca2 [id=6187d80c67d54a229a0868f99c892896]) successfully deleted
I20260926 19:58:36.142843 13013 raft_consensus.cc:385] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.143097 13013 raft_consensus.cc:740] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.144821 13032 ts_tablet_manager.cc:1929] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:36.145651 13032 log.cc:1199] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/5b229bf63055460dbb6b6df6f3c55887
I20260926 19:58:36.146482 13032 ts_tablet_manager.cc:1950] T 5b229bf63055460dbb6b6df6f3c55887 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:36.148818 13014 tablet_bootstrap.cc:492] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:36.149674 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 5b229bf63055460dbb6b6df6f3c55887 (table default.Table_1c090c26a87e499baeb08c7be0703ca2 [id=6187d80c67d54a229a0868f99c892896]) successfully deleted
I20260926 19:58:36.149431 13013 consensus_queue.cc:260] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.152261 13013 ts_tablet_manager.cc:1434] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.016s user 0.006s sys 0.002s
I20260926 19:58:36.160584 13012 tablet_bootstrap.cc:654] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.163476 13014 tablet_bootstrap.cc:654] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.252799 13014 tablet_bootstrap.cc:492] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:36.253234 13014 ts_tablet_manager.cc:1403] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.105s user 0.050s sys 0.044s
I20260926 19:58:36.255750 13014 raft_consensus.cc:359] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.256425 13014 raft_consensus.cc:385] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.256688 13014 raft_consensus.cc:740] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.257364 13014 consensus_queue.cc:260] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.260245 13014 ts_tablet_manager.cc:1434] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260926 19:58:36.261140 13014 tablet_bootstrap.cc:492] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:36.267515 13014 tablet_bootstrap.cc:654] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.279597 13014 tablet_bootstrap.cc:492] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:36.280484 13014 ts_tablet_manager.cc:1403] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.005s
I20260926 19:58:36.283975 13014 raft_consensus.cc:359] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.284631 13014 raft_consensus.cc:385] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.284881 13014 raft_consensus.cc:740] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.285565 13014 consensus_queue.cc:260] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.288863 13014 ts_tablet_manager.cc:1434] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260926 19:58:36.289772 13014 tablet_bootstrap.cc:492] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:36.296375 13014 tablet_bootstrap.cc:654] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.311923 13014 tablet_bootstrap.cc:492] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:36.312309 13014 ts_tablet_manager.cc:1403] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.011s
I20260926 19:58:36.314517 13014 raft_consensus.cc:359] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.315140 13014 raft_consensus.cc:385] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.315377 13014 raft_consensus.cc:740] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.316032 13014 consensus_queue.cc:260] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.317975 13014 ts_tablet_manager.cc:1434] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 19:58:36.320765 13012 tablet_bootstrap.cc:492] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:36.321187 13012 ts_tablet_manager.cc:1403] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.283s user 0.008s sys 0.007s
I20260926 19:58:36.323730 13012 raft_consensus.cc:359] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.326833 13012 raft_consensus.cc:385] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.327107 13012 raft_consensus.cc:740] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.327876 13012 consensus_queue.cc:260] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.329917 13012 ts_tablet_manager.cc:1434] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260926 19:58:36.331296 13012 tablet_bootstrap.cc:492] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:36.337708 13012 tablet_bootstrap.cc:654] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.341236 12395 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_73148470e8e44a98880dc32126902294" } modify_external_catalogs: true
I20260926 19:58:36.342079 12395 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_73148470e8e44a98880dc32126902294" } modify_external_catalogs: true
I20260926 19:58:36.352979 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet de292b22227a440fb2dc9e9a11f208c2
I20260926 19:58:36.358199 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet de292b22227a440fb2dc9e9a11f208c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:36.362243 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet de292b22227a440fb2dc9e9a11f208c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:36.362497 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet de292b22227a440fb2dc9e9a11f208c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de292b22227a440fb2dc9e9a11f208c2 already in progress: creating tablet
I20260926 19:58:36.365177 13012 tablet_bootstrap.cc:492] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:36.365638 13012 ts_tablet_manager.cc:1403] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.012s
I20260926 19:58:36.368050 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet de292b22227a440fb2dc9e9a11f208c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:36.373306 13031 tablet_replica.cc:333] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:36.374130 13031 raft_consensus.cc:2243] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:36.374614 13031 raft_consensus.cc:2272] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.376843 13031 ts_tablet_manager.cc:1916] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.378206 13060 tablet_replica.cc:333] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:36.381129 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_3b6f2ff3ca9c4856a0fd107c047c2633"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:36.380867 13012 raft_consensus.cc:359] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.383896 13012 raft_consensus.cc:385] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.384315 13012 raft_consensus.cc:740] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.385485 13012 consensus_queue.cc:260] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:36.383504 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3b6f2ff3ca9c4856a0fd107c047c2633 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:36.397208 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ec253204a23a44bda812070e8c57cc39 has no consensus state
I20260926 19:58:36.399816 13060 raft_consensus.cc:2243] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:36.400490 13060 raft_consensus.cc:2272] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.401702 13012 ts_tablet_manager.cc:1434] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.036s user 0.009s sys 0.000s
I20260926 19:58:36.406062 13060 ts_tablet_manager.cc:1916] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.413496 13031 ts_tablet_manager.cc:1929] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:36.415562 13031 log.cc:1199] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/de292b22227a440fb2dc9e9a11f208c2
I20260926 19:58:36.416226 13031 ts_tablet_manager.cc:1950] T de292b22227a440fb2dc9e9a11f208c2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:36.418399 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_cfb0abe6db43480496837ee334f94e25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:36.419065 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet de292b22227a440fb2dc9e9a11f208c2 (table default.Table_ab469173bd8d4ff197f4240a8a86fd71 [id=27fa522794c84e1f96b15a4bb6b5570b]) successfully deleted
W20260926 19:58:36.420107 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfb0abe6db43480496837ee334f94e25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:36.421638 13012 tablet_bootstrap.cc:492] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:36.432156 13012 tablet_bootstrap.cc:654] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.433986 13060 ts_tablet_manager.cc:1929] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:36.434856 13060 log.cc:1199] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/de292b22227a440fb2dc9e9a11f208c2
I20260926 19:58:36.435727 13060 ts_tablet_manager.cc:1950] T de292b22227a440fb2dc9e9a11f208c2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:36.440284 12902 tablet_service.cc:1511] Processing CreateTablet for tablet ec253204a23a44bda812070e8c57cc39 (DEFAULT_TABLE table=default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.440558 12740 tablet_service.cc:1511] Processing CreateTablet for tablet ec253204a23a44bda812070e8c57cc39 (DEFAULT_TABLE table=default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.441721 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec253204a23a44bda812070e8c57cc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.442716 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet de292b22227a440fb2dc9e9a11f208c2 (table default.Table_ab469173bd8d4ff197f4240a8a86fd71 [id=27fa522794c84e1f96b15a4bb6b5570b]) successfully deleted
I20260926 19:58:36.444144 12567 tablet_service.cc:1511] Processing CreateTablet for tablet ec253204a23a44bda812070e8c57cc39 (DEFAULT_TABLE table=default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.441390 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec253204a23a44bda812070e8c57cc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.445708 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec253204a23a44bda812070e8c57cc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.461081 13014 tablet_bootstrap.cc:492] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:36.461077 13012 tablet_bootstrap.cc:492] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:36.461498 13012 ts_tablet_manager.cc:1403] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.040s user 0.016s sys 0.005s
I20260926 19:58:36.464140 13012 raft_consensus.cc:359] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.465248 13012 raft_consensus.cc:385] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.465538 13012 raft_consensus.cc:740] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.466684 13014 tablet_bootstrap.cc:654] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.473805 13012 consensus_queue.cc:260] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.486893 13012 ts_tablet_manager.cc:1434] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260926 19:58:36.494225 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9ac55a637c1e41f0bfca50930bd701c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:36.495926 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ac55a637c1e41f0bfca50930bd701c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:36.499051 12390 catalog_manager.cc:5838] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:36.500432 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 2 got 0
I20260926 19:58:36.501682 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 4e706596959e400398bf2213653f06c9 (table default.Table_0ce88298a7c04aebabf03fcecc58bd5a [id=e2cf63902d2c4d28815fca606eef0274]). Expected version 1 got 0
I20260926 19:58:36.502382 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 8538b46e783f4ffb9a31c12939f954af (table default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]). Expected version 1 got 0
I20260926 19:58:36.503077 12395 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet a288f85f189f4b7baca505928471bb68
I20260926 19:58:36.506455 13012 tablet_bootstrap.cc:492] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:36.509812 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet a288f85f189f4b7baca505928471bb68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:36.510634 13060 tablet_replica.cc:333] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:36.511458 13060 raft_consensus.cc:2243] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:36.512205 13060 pending_rounds.cc:70] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:36.512502 13060 pending_rounds.cc:77] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694325170315264 op_type: NO_OP noop_request { }
I20260926 19:58:36.512933 13060 raft_consensus.cc:2889] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:36.513267 13060 raft_consensus.cc:2272] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.514856 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet a288f85f189f4b7baca505928471bb68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:36.515197 13039 consensus_queue.cc:1048] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:36.515872 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:36.516569 13013 tablet_bootstrap.cc:492] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:36.516885 13031 tablet_replica.cc:333] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:36.518086 13031 raft_consensus.cc:2243] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:36.518728 13031 raft_consensus.cc:2272] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.523469 13060 ts_tablet_manager.cc:1916] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.525329 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet a288f85f189f4b7baca505928471bb68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:36.526142 13032 tablet_replica.cc:333] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:36.526870 13032 raft_consensus.cc:2243] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:36.527045 13013 tablet_bootstrap.cc:654] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.527325 13032 raft_consensus.cc:2272] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.529381 13032 ts_tablet_manager.cc:1916] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.521373 13031 ts_tablet_manager.cc:1916] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.540251 13012 tablet_bootstrap.cc:654] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.547467 13014 tablet_bootstrap.cc:492] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:36.547911 13014 ts_tablet_manager.cc:1403] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.087s user 0.019s sys 0.010s
I20260926 19:58:36.552695 13014 raft_consensus.cc:359] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.553325 13014 raft_consensus.cc:385] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.553620 13014 raft_consensus.cc:740] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.554409 13014 consensus_queue.cc:260] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.563741 12388 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b1e07cc33f1143909a84c80f7d9ad5c5" } modify_external_catalogs: true
I20260926 19:58:36.564257 12388 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b1e07cc33f1143909a84c80f7d9ad5c5" } modify_external_catalogs: true
I20260926 19:58:36.588451 13014 ts_tablet_manager.cc:1434] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.038s user 0.004s sys 0.000s
I20260926 19:58:36.597721 13031 ts_tablet_manager.cc:1929] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:36.598592 13031 log.cc:1199] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/a288f85f189f4b7baca505928471bb68
I20260926 19:58:36.601234 13031 ts_tablet_manager.cc:1950] T a288f85f189f4b7baca505928471bb68 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:36.603531 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet d6166f0b9fe24ceb8e83b9910651d156
I20260926 19:58:36.605474 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet d6166f0b9fe24ceb8e83b9910651d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:36.605873 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet d6166f0b9fe24ceb8e83b9910651d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:36.611129 13013 tablet_bootstrap.cc:492] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:36.611536 13013 ts_tablet_manager.cc:1403] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.095s user 0.012s sys 0.004s
I20260926 19:58:36.613956 13013 raft_consensus.cc:359] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.614578 13013 raft_consensus.cc:385] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.614818 13013 raft_consensus.cc:740] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.615603 13013 consensus_queue.cc:260] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.620910 13013 ts_tablet_manager.cc:1434] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260926 19:58:36.620692 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet a288f85f189f4b7baca505928471bb68 (table default.Table_73148470e8e44a98880dc32126902294 [id=4b885f1ea09a42caa59023689fcc46f9]) successfully deleted
I20260926 19:58:36.624029 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_45d30c1ab9414653bfb03865e36d16b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:36.624642 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3b6f2ff3ca9c4856a0fd107c047c2633" } new_table_name: "default.Table_fc8850cc55f844679e12c2e9878cf1b0" modify_external_catalogs: true
I20260926 19:58:36.625088 13026 raft_consensus.cc:493] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:36.625806 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45d30c1ab9414653bfb03865e36d16b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:36.625491 13026 raft_consensus.cc:515] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.629858 13026 leader_election.cc:290] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:36.632220 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8538b46e783f4ffb9a31c12939f954af" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:36.632972 12923 raft_consensus.cc:2468] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:36.634954 12521 leader_election.cc:304] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:36.638906 13026 consensus_queue.cc:1048] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:36.639902 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8538b46e783f4ffb9a31c12939f954af" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:36.640794 13026 raft_consensus.cc:2804] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:36.640920 12760 raft_consensus.cc:2468] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:36.641120 13026 raft_consensus.cc:493] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:36.641409 13026 raft_consensus.cc:3060] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:36.655225 12924 raft_consensus.cc:1275] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Refusing update from remote peer f6e9d6439fd54b28925a68a6145d3dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260926 19:58:36.655893 12395 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4eca1e5734324f99b9362915bdd00239 has no consensus state
I20260926 19:58:36.662870 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet d6166f0b9fe24ceb8e83b9910651d156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:36.665972 13039 consensus_queue.cc:1048] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:36.667569 13031 tablet_replica.cc:333] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:36.667759 12760 raft_consensus.cc:1275] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Refusing update from remote peer f6e9d6439fd54b28925a68a6145d3dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:36.673197 13076 consensus_queue.cc:1048] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260926 19:58:36.674921 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 5b229bf63055460dbb6b6df6f3c55887 because the tablet was not found. No further retry: Not found: Tablet not found: 5b229bf63055460dbb6b6df6f3c55887
I20260926 19:58:36.681169 13032 ts_tablet_manager.cc:1929] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:36.681571 13032 log.cc:1199] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/a288f85f189f4b7baca505928471bb68
I20260926 19:58:36.682251 13032 ts_tablet_manager.cc:1950] T a288f85f189f4b7baca505928471bb68 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:36.692384 13012 tablet_bootstrap.cc:492] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:36.692618 13031 raft_consensus.cc:2243] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:36.693084 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_c7b5800561444fd7ad044784765fd2bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:36.694815 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7b5800561444fd7ad044784765fd2bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:36.702355 12391 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 4e706596959e400398bf2213653f06c9 (table default.Table_0ce88298a7c04aebabf03fcecc58bd5a [id=e2cf63902d2c4d28815fca606eef0274]). Expected version 1 got 0
I20260926 19:58:36.702983 12391 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 8538b46e783f4ffb9a31c12939f954af (table default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]). Expected version 1 got 0
I20260926 19:58:36.704347 13031 raft_consensus.cc:2272] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.705112 13012 ts_tablet_manager.cc:1403] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.199s user 0.013s sys 0.000s
I20260926 19:58:36.706368 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 4e706596959e400398bf2213653f06c9 (table default.Table_0ce88298a7c04aebabf03fcecc58bd5a [id=e2cf63902d2c4d28815fca606eef0274]). Expected version 1 got 0
I20260926 19:58:36.707250 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 8538b46e783f4ffb9a31c12939f954af (table default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]). Expected version 1 got 0
I20260926 19:58:36.708226 13012 raft_consensus.cc:359] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.708818 13012 raft_consensus.cc:385] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:58:36.708755 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e706596959e400398bf2213653f06c9 consensus state has no leader
I20260926 19:58:36.709046 13012 raft_consensus.cc:740] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.711812 13031 ts_tablet_manager.cc:1916] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.713917 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet a288f85f189f4b7baca505928471bb68 (table default.Table_73148470e8e44a98880dc32126902294 [id=4b885f1ea09a42caa59023689fcc46f9]) successfully deleted
I20260926 19:58:36.725782 13012 consensus_queue.cc:260] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.733711 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet a288f85f189f4b7baca505928471bb68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:36.733686 13012 ts_tablet_manager.cc:1434] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260926 19:58:36.738554 13032 tablet_replica.cc:333] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:36.742770 13012 tablet_bootstrap.cc:492] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:36.744082 12390 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet ec253204a23a44bda812070e8c57cc39 (table default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]). Expected version 1 got 0
I20260926 19:58:36.748832 12391 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_55128b0718b349d18a8f378ecd6d0a24" } modify_external_catalogs: true
I20260926 19:58:36.754139 13032 raft_consensus.cc:2243] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:36.754222 12391 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_55128b0718b349d18a8f378ecd6d0a24" } modify_external_catalogs: true
I20260926 19:58:36.754750 13032 raft_consensus.cc:2272] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.750149 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_3a61927510f1455ead6604257dcf158c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:36.751240 12381 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.254:33143, user_credentials={real_user=slave}} blocked reactor thread for 46959.7us
I20260926 19:58:36.760985 13012 tablet_bootstrap.cc:654] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:36.761489 13032 ts_tablet_manager.cc:1916] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.764577 12740 tablet_service.cc:1511] Processing CreateTablet for tablet ad42626eb2054e7fb549f1c4d20b4cfc (DEFAULT_TABLE table=default.Table_55128b0718b349d18a8f378ecd6d0a24 [id=7606ba867d204bbcb4dc88918435fb1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.765671 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad42626eb2054e7fb549f1c4d20b4cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.769305 13031 ts_tablet_manager.cc:1929] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:36.769755 13031 log.cc:1199] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/d6166f0b9fe24ceb8e83b9910651d156
I20260926 19:58:36.770539 13031 ts_tablet_manager.cc:1950] T d6166f0b9fe24ceb8e83b9910651d156 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:36.774545 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 2 got 0
I20260926 19:58:36.785804 13012 tablet_bootstrap.cc:492] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:36.786221 13012 ts_tablet_manager.cc:1403] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.002s
I20260926 19:58:36.786264 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet de292b22227a440fb2dc9e9a11f208c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:36.789069 13012 raft_consensus.cc:359] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.789942 13012 raft_consensus.cc:385] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:36.790479 13012 raft_consensus.cc:740] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:36.791117 13012 consensus_queue.cc:260] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.795946 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_d2d87877a7c4476aa7e41623c09ed4df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:36.798833 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2d87877a7c4476aa7e41623c09ed4df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:36.801592 13012 ts_tablet_manager.cc:1434] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20260926 19:58:36.807793 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet a288f85f189f4b7baca505928471bb68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:33594
W20260926 19:58:36.759024 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a61927510f1455ead6604257dcf158c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:36.823071 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_b1e07cc33f1143909a84c80f7d9ad5c5 [id=418957ebf6b5490abef44b6064f7154a]) successfully deleted
I20260926 19:58:36.828128 13032 ts_tablet_manager.cc:1929] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:36.828531 13032 log.cc:1199] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/d6166f0b9fe24ceb8e83b9910651d156
I20260926 19:58:36.829241 13032 ts_tablet_manager.cc:1950] T d6166f0b9fe24ceb8e83b9910651d156 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
W20260926 19:58:36.833244 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet a288f85f189f4b7baca505928471bb68 because the tablet was not found. No further retry: Not found: Tablet not found: a288f85f189f4b7baca505928471bb68
I20260926 19:58:36.845343 13060 ts_tablet_manager.cc:1929] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:36.845764 13060 log.cc:1199] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/a288f85f189f4b7baca505928471bb68
I20260926 19:58:36.846506 13060 ts_tablet_manager.cc:1950] T a288f85f189f4b7baca505928471bb68 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:36.849581 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_b1e07cc33f1143909a84c80f7d9ad5c5 [id=418957ebf6b5490abef44b6064f7154a]) successfully deleted
I20260926 19:58:36.850564 12903 tablet_service.cc:1511] Processing CreateTablet for tablet ad42626eb2054e7fb549f1c4d20b4cfc (DEFAULT_TABLE table=default.Table_55128b0718b349d18a8f378ecd6d0a24 [id=7606ba867d204bbcb4dc88918435fb1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.850955 13032 tablet_replica.cc:333] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:36.851663 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad42626eb2054e7fb549f1c4d20b4cfc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:36.852615 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet a288f85f189f4b7baca505928471bb68 because the tablet was not found. No further retry: Not found: Tablet not found: a288f85f189f4b7baca505928471bb68
I20260926 19:58:36.853932 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet ec253204a23a44bda812070e8c57cc39 (table default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]). Expected version 1 got 0
W20260926 19:58:36.855685 12519 consensus_peers.cc:597] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6166f0b9fe24ceb8e83b9910651d156. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:36.851694 13032 raft_consensus.cc:2243] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:36.866284 13032 raft_consensus.cc:2272] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.869118 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet de292b22227a440fb2dc9e9a11f208c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:36.857245 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b229bf63055460dbb6b6df6f3c55887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:35 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:36.871325 12565 tablet_service.cc:1511] Processing CreateTablet for tablet ad42626eb2054e7fb549f1c4d20b4cfc (DEFAULT_TABLE table=default.Table_55128b0718b349d18a8f378ecd6d0a24 [id=7606ba867d204bbcb4dc88918435fb1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.871950 13032 ts_tablet_manager.cc:1916] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:36.872462 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad42626eb2054e7fb549f1c4d20b4cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.883721 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet ec253204a23a44bda812070e8c57cc39 (table default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]). Expected version 1 got 0
I20260926 19:58:36.884835 13069 mvcc.cc:204] Tried to move back new op lower bound from 7333694327346348032 to 7333694323858198528. Current Snapshot: MvccSnapshot[applied={T|T < 7333694327346348032}]
I20260926 19:58:36.897909 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 05a6d163900947689b24489787bc119c (DEFAULT_TABLE table=default.Table_68d42bf187a146958c5daf317378dafd [id=37adf5d8d6c1411b8e6028df63dcd641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.900528 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05a6d163900947689b24489787bc119c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.912923 13068 mvcc.cc:204] Tried to move back new op lower bound from 7333694327346348032 to 7333694323858198528. Current Snapshot: MvccSnapshot[applied={T|T < 7333694327346348032}]
I20260926 19:58:36.912781 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_60d74638cb464ebd99d13cfed16bc764" } new_table_name: "default.Table_1b0fc924e4754830b2e296f3835dd9d2" modify_external_catalogs: true
I20260926 19:58:36.918118 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f6cb69a2397348c3a00214884a88d013" } modify_external_catalogs: true
W20260926 19:58:36.918936 12521 consensus_peers.cc:597] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6166f0b9fe24ceb8e83b9910651d156. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:36.919062 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f6cb69a2397348c3a00214884a88d013" } modify_external_catalogs: true
I20260926 19:58:36.893227 12391 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet ad42626eb2054e7fb549f1c4d20b4cfc
I20260926 19:58:36.929386 13060 tablet_replica.cc:333] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:36.930358 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 05a6d163900947689b24489787bc119c (DEFAULT_TABLE table=default.Table_68d42bf187a146958c5daf317378dafd [id=37adf5d8d6c1411b8e6028df63dcd641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.930433 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet a288f85f189f4b7baca505928471bb68 (table default.Table_73148470e8e44a98880dc32126902294 [id=4b885f1ea09a42caa59023689fcc46f9]) successfully deleted
I20260926 19:58:36.931922 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05a6d163900947689b24489787bc119c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.934178 13060 raft_consensus.cc:2243] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:36.934983 13032 ts_tablet_manager.cc:1929] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:36.935216 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet ad42626eb2054e7fb549f1c4d20b4cfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:36.935381 13032 log.cc:1199] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/de292b22227a440fb2dc9e9a11f208c2
I20260926 19:58:36.938464 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet ad42626eb2054e7fb549f1c4d20b4cfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:36.939229 13032 ts_tablet_manager.cc:1950] T de292b22227a440fb2dc9e9a11f208c2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:36.933694 13083 tablet.cc:1751] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:36.942160 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 05a6d163900947689b24489787bc119c (DEFAULT_TABLE table=default.Table_68d42bf187a146958c5daf317378dafd [id=37adf5d8d6c1411b8e6028df63dcd641]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.943836 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05a6d163900947689b24489787bc119c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.951053 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet ad42626eb2054e7fb549f1c4d20b4cfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:36.954926 13026 raft_consensus.cc:515] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.956789 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet de292b22227a440fb2dc9e9a11f208c2 (table default.Table_ab469173bd8d4ff197f4240a8a86fd71 [id=27fa522794c84e1f96b15a4bb6b5570b]) successfully deleted
I20260926 19:58:36.957681 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8538b46e783f4ffb9a31c12939f954af" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
W20260926 19:58:36.957950 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet ad42626eb2054e7fb549f1c4d20b4cfc because the tablet was not found. No further retry: Not found: Tablet not found: ad42626eb2054e7fb549f1c4d20b4cfc
I20260926 19:58:36.958251 12761 raft_consensus.cc:3060] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:36.959098 13026 leader_election.cc:290] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:36.960790 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8538b46e783f4ffb9a31c12939f954af" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:36.961339 12924 raft_consensus.cc:3060] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:36.965039 13018 raft_consensus.cc:493] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:36.965520 13018 raft_consensus.cc:515] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:36.966681 12761 raft_consensus.cc:2468] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:36.967536 13018 leader_election.cc:290] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:36.968555 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e706596959e400398bf2213653f06c9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:36.968890 12519 leader_election.cc:304] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:36.969550 12595 raft_consensus.cc:2468] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:36.971808 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e706596959e400398bf2213653f06c9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:36.972702 12922 raft_consensus.cc:2468] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:36.974220 12695 leader_election.cc:304] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:36.975152 13018 raft_consensus.cc:2804] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:36.975456 13018 raft_consensus.cc:493] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:36.975747 13018 raft_consensus.cc:3060] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:36.983407 12924 raft_consensus.cc:2468] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:36.985548 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 82f3de5e9b6641508ba56f9b33e930f2 (DEFAULT_TABLE table=default.Table_26612903c7e7487bb95439ebde7b204e [id=f53ec6e521114c0c9cc60e7627db131c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:36.989750 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82f3de5e9b6641508ba56f9b33e930f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:36.991376 13060 pending_rounds.cc:70] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:36.991642 13060 pending_rounds.cc:77] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333694327391436800 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "d6166f0b9fe24ceb8e83b9910651d156" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_b1e07cc33f1143909a84c80f7d9ad5c5" dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" new_extra_config { } }
W20260926 19:58:36.992542 13060 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:37792: Aborted: Op aborted
I20260926 19:58:36.994515 13060 raft_consensus.cc:2272] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:36.997219 13060 ts_tablet_manager.cc:1916] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.001641 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_36db24f73aca410799bcb881e2bd1bf9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:37.004454 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36db24f73aca410799bcb881e2bd1bf9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:37.014307 13026 raft_consensus.cc:2804] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:37.018101 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 82f3de5e9b6641508ba56f9b33e930f2 (DEFAULT_TABLE table=default.Table_26612903c7e7487bb95439ebde7b204e [id=f53ec6e521114c0c9cc60e7627db131c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.019398 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82f3de5e9b6641508ba56f9b33e930f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.019824 13018 raft_consensus.cc:515] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:37.022624 13018 leader_election.cc:290] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:37.026293 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e706596959e400398bf2213653f06c9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:37.026880 12595 raft_consensus.cc:3060] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.037705 12595 raft_consensus.cc:2468] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:37.038676 13026 raft_consensus.cc:697] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:37.039325 12695 leader_election.cc:304] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:37.039484 13026 consensus_queue.cc:237] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:37.040738 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e706596959e400398bf2213653f06c9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:37.047356 13081 tablet.cc:1751] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:37.041337 12924 raft_consensus.cc:3060] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.070621 12738 tablet_service.cc:1511] Processing CreateTablet for tablet e20905f83b2e4c7fa409fce38ec5f81e (DEFAULT_TABLE table=default.Table_a6745a20e46f46c8bda622354e9d558b [id=c699605165e847f4bca3d95b659eb989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.071848 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20905f83b2e4c7fa409fce38ec5f81e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:36.954996 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet ad42626eb2054e7fb549f1c4d20b4cfc because the tablet was not found. No further retry: Not found: Tablet not found: ad42626eb2054e7fb549f1c4d20b4cfc
W20260926 19:58:37.053838 12381 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 4eca1e5734324f99b9362915bdd00239 (table default.Table_fc8850cc55f844679e12c2e9878cf1b0 [id=4692b08d69b344cdaf9226c62886da70], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4eca1e5734324f99b9362915bdd00239 has no consensus state
I20260926 19:58:37.058293 12388 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 2 got 0
I20260926 19:58:37.079815 13039 raft_consensus.cc:493] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:37.080281 13039 raft_consensus.cc:515] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:37.082223 13039 leader_election.cc:290] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:37.087178 13013 tablet_bootstrap.cc:492] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:37.100193 13013 tablet_bootstrap.cc:654] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.109894 13060 ts_tablet_manager.cc:1929] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:37.110751 13060 log.cc:1199] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/d6166f0b9fe24ceb8e83b9910651d156
I20260926 19:58:37.111460 13060 ts_tablet_manager.cc:1950] T d6166f0b9fe24ceb8e83b9910651d156 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:37.126573 13060 tablet_replica.cc:333] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:37.127302 13060 raft_consensus.cc:2243] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.127864 13060 raft_consensus.cc:2272] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.131263 13013 tablet_bootstrap.cc:492] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:37.131662 13013 ts_tablet_manager.cc:1403] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.045s user 0.010s sys 0.000s
I20260926 19:58:37.136765 13060 ts_tablet_manager.cc:1916] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.140517 12395 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6] alter complete (version 2)
I20260926 19:58:37.143667 13013 raft_consensus.cc:359] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.144371 13013 raft_consensus.cc:385] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.144619 13013 raft_consensus.cc:740] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.145205 13013 consensus_queue.cc:260] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.147003 13013 ts_tablet_manager.cc:1434] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260926 19:58:37.147802 13013 tablet_bootstrap.cc:492] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:37.154518 13013 tablet_bootstrap.cc:654] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:37.158497 12382 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 203563us
I20260926 19:58:37.161900 13060 ts_tablet_manager.cc:1929] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.162272 13060 log.cc:1199] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/5b229bf63055460dbb6b6df6f3c55887
I20260926 19:58:37.163427 13060 ts_tablet_manager.cc:1950] T 5b229bf63055460dbb6b6df6f3c55887 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:37.163870 13013 tablet_bootstrap.cc:492] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:37.164458 13013 ts_tablet_manager.cc:1403] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.001s
I20260926 19:58:37.165877 13060 tablet_replica.cc:333] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:37.166640 13060 raft_consensus.cc:2243] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.167088 13060 raft_consensus.cc:2272] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.167903 13013 raft_consensus.cc:359] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.168627 13013 raft_consensus.cc:385] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.168919 13013 raft_consensus.cc:740] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.169497 13060 ts_tablet_manager.cc:1916] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.169543 13013 consensus_queue.cc:260] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.178066 13012 tablet_bootstrap.cc:492] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:37.179589 13013 ts_tablet_manager.cc:1434] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.015s user 0.003s sys 0.002s
I20260926 19:58:37.183282 13012 tablet_bootstrap.cc:654] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.186419 13060 ts_tablet_manager.cc:1929] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.186802 13060 log.cc:1199] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/ad42626eb2054e7fb549f1c4d20b4cfc
I20260926 19:58:37.187424 13060 ts_tablet_manager.cc:1950] T ad42626eb2054e7fb549f1c4d20b4cfc P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:37.189623 13018 raft_consensus.cc:2804] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:37.209122 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 82f3de5e9b6641508ba56f9b33e930f2 (DEFAULT_TABLE table=default.Table_26612903c7e7487bb95439ebde7b204e [id=f53ec6e521114c0c9cc60e7627db131c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.210675 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82f3de5e9b6641508ba56f9b33e930f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.217268 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec253204a23a44bda812070e8c57cc39" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:37.218003 12760 raft_consensus.cc:2468] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:37.218292 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet d6166f0b9fe24ceb8e83b9910651d156 (table default.Table_b1e07cc33f1143909a84c80f7d9ad5c5 [id=418957ebf6b5490abef44b6064f7154a]) successfully deleted
W20260926 19:58:37.219206 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet de292b22227a440fb2dc9e9a11f208c2 because the tablet was not found. No further retry: Not found: Tablet not found: de292b22227a440fb2dc9e9a11f208c2
I20260926 19:58:37.223776 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 5b229bf63055460dbb6b6df6f3c55887 (table default.Table_1c090c26a87e499baeb08c7be0703ca2 [id=6187d80c67d54a229a0868f99c892896]) successfully deleted
I20260926 19:58:37.224594 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet ad42626eb2054e7fb549f1c4d20b4cfc (table default.Table_55128b0718b349d18a8f378ecd6d0a24 [id=7606ba867d204bbcb4dc88918435fb1a]) successfully deleted
I20260926 19:58:37.225371 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec253204a23a44bda812070e8c57cc39" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:37.226022 12923 raft_consensus.cc:2468] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:37.225935 12568 tablet_service.cc:1511] Processing CreateTablet for tablet e20905f83b2e4c7fa409fce38ec5f81e (DEFAULT_TABLE table=default.Table_a6745a20e46f46c8bda622354e9d558b [id=c699605165e847f4bca3d95b659eb989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.227340 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20905f83b2e4c7fa409fce38ec5f81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.232055 12519 leader_election.cc:304] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:37.239888 12924 raft_consensus.cc:2468] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:37.241614 12396 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 2 got 0
W20260926 19:58:37.245361 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e706596959e400398bf2213653f06c9 consensus state has no leader
I20260926 19:58:37.247334 13092 tablet_bootstrap.cc:492] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:37.253489 13092 tablet_bootstrap.cc:654] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.255717 13012 tablet_bootstrap.cc:492] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:37.256667 13039 raft_consensus.cc:2804] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:37.257359 13039 raft_consensus.cc:493] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:37.257788 13039 raft_consensus.cc:3060] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.256151 13012 ts_tablet_manager.cc:1403] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.078s user 0.021s sys 0.022s
I20260926 19:58:37.263440 13012 raft_consensus.cc:359] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.264086 13012 raft_consensus.cc:385] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.264346 13012 raft_consensus.cc:740] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.265141 13039 raft_consensus.cc:515] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:37.269618 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet ec253204a23a44bda812070e8c57cc39
I20260926 19:58:37.271759 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet ec253204a23a44bda812070e8c57cc39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:37.271601 12565 tablet_service.cc:1558] Processing DeleteTablet for tablet ec253204a23a44bda812070e8c57cc39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:37.272938 13060 tablet_replica.cc:333] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:37.274013 13060 raft_consensus.cc:2243] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.274474 13060 raft_consensus.cc:2272] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.261667 13075 mvcc.cc:204] Tried to move back new op lower bound from 7333694327346348032 to 7333694323858198528. Current Snapshot: MvccSnapshot[applied={T|T < 7333694327346348032}]
I20260926 19:58:37.285707 13039 leader_election.cc:290] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:37.273799 13012 consensus_queue.cc:260] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.287001 13093 tablet.cc:1751] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:37.289875 13060 ts_tablet_manager.cc:1916] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.297302 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec253204a23a44bda812070e8c57cc39" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:37.297883 12924 raft_consensus.cc:3060] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.301007 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec253204a23a44bda812070e8c57cc39" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:37.303009 12390 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_ee0eba1e9d7548799bca25f3815c5ad3 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]). Expected version 2 got 0
I20260926 19:58:37.304198 13013 tablet_bootstrap.cc:492] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:37.304736 12760 raft_consensus.cc:3060] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.304855 12924 raft_consensus.cc:2468] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:37.306121 12521 leader_election.cc:304] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
W20260926 19:58:37.306859 12521 raft_consensus.cc:2686] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 19:58:37.309979 13013 tablet_bootstrap.cc:654] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.310700 13032 tablet_replica.cc:333] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:37.317847 12760 raft_consensus.cc:2468] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:37.318339 13032 raft_consensus.cc:2243] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.320322 13032 raft_consensus.cc:2272] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.320719 13013 tablet_bootstrap.cc:492] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:37.321578 13013 ts_tablet_manager.cc:1403] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.008s
I20260926 19:58:37.322738 13032 ts_tablet_manager.cc:1916] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.324055 13013 raft_consensus.cc:359] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.324695 13013 raft_consensus.cc:385] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.324954 13013 raft_consensus.cc:740] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.337937 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_fc8850cc55f844679e12c2e9878cf1b0" } new_table_name: "default.Table_dc1bdf9e02c44b0bb32ccf1e18872c04" modify_external_catalogs: true
I20260926 19:58:37.338366 12903 tablet_service.cc:1511] Processing CreateTablet for tablet e20905f83b2e4c7fa409fce38ec5f81e (DEFAULT_TABLE table=default.Table_a6745a20e46f46c8bda622354e9d558b [id=c699605165e847f4bca3d95b659eb989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.339797 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e20905f83b2e4c7fa409fce38ec5f81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.342850 13018 raft_consensus.cc:697] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:37.343808 13018 consensus_queue.cc:237] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:37.348219 13013 consensus_queue.cc:260] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.351826 13060 ts_tablet_manager.cc:1929] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.352278 13060 log.cc:1199] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/ec253204a23a44bda812070e8c57cc39
I20260926 19:58:37.353175 13060 ts_tablet_manager.cc:1950] T ec253204a23a44bda812070e8c57cc39 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:37.356727 13013 ts_tablet_manager.cc:1434] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.035s user 0.004s sys 0.001s
I20260926 19:58:37.357892 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_cba4802138f54b36a76988eddd698d94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:37.359675 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cba4802138f54b36a76988eddd698d94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:37.360988 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet ec253204a23a44bda812070e8c57cc39 (table default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]) successfully deleted
I20260926 19:58:37.361250 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9729e85fef9b43b7bc6f5a034c04e20e" } modify_external_catalogs: true
I20260926 19:58:37.362064 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_170a499a114147fc9650b978b327f7e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:37.363724 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_170a499a114147fc9650b978b327f7e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:37.364993 13013 tablet_bootstrap.cc:492] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:37.365896 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9729e85fef9b43b7bc6f5a034c04e20e" } modify_external_catalogs: true
I20260926 19:58:37.381441 13013 tablet_bootstrap.cc:654] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.402602 13013 tablet_bootstrap.cc:492] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:37.403012 13013 ts_tablet_manager.cc:1403] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.038s user 0.012s sys 0.000s
I20260926 19:58:37.405428 13013 raft_consensus.cc:359] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.406075 13013 raft_consensus.cc:385] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.406330 13013 raft_consensus.cc:740] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.410267 13039 raft_consensus.cc:493] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:37.410706 13039 raft_consensus.cc:515] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.411882 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet ec253204a23a44bda812070e8c57cc39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:37.412683 13039 leader_election.cc:290] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:37.413442 13031 tablet_replica.cc:333] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:37.414276 13031 raft_consensus.cc:2243] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.414757 13031 raft_consensus.cc:2272] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.415653 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 4e82262ea1c441808e9765aef911c903 (DEFAULT_TABLE table=default.Table_c7b5800561444fd7ad044784765fd2bb [id=68ea017f68a5467c8c1d1c1ea98f4701]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.416522 13031 ts_tablet_manager.cc:1916] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.415364 12390 catalog_manager.cc:5838] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:37.417256 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e82262ea1c441808e9765aef911c903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.417296 12390 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 4e706596959e400398bf2213653f06c9 (table default.Table_1b0fc924e4754830b2e296f3835dd9d2 [id=e2cf63902d2c4d28815fca606eef0274]). Expected version 3 got 0
I20260926 19:58:37.417882 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f3de5e9b6641508ba56f9b33e930f2" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:37.419476 12519 leader_election.cc:343] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:37.419859 13013 consensus_queue.cc:260] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.423748 13012 ts_tablet_manager.cc:1434] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.163s user 0.011s sys 0.010s
I20260926 19:58:37.424798 13012 tablet_bootstrap.cc:492] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:37.432238 12521 leader_election.cc:343] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Not found: Tablet not found: 82f3de5e9b6641508ba56f9b33e930f2
I20260926 19:58:37.432765 12521 leader_election.cc:304] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:37.433772 13012 tablet_bootstrap.cc:654] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.439805 13013 ts_tablet_manager.cc:1434] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.037s user 0.004s sys 0.003s
I20260926 19:58:37.440222 13039 raft_consensus.cc:2749] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:37.444584 13032 ts_tablet_manager.cc:1929] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.448724 13032 log.cc:1199] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/ec253204a23a44bda812070e8c57cc39
I20260926 19:58:37.451318 13032 ts_tablet_manager.cc:1950] T ec253204a23a44bda812070e8c57cc39 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:37.454319 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet ec253204a23a44bda812070e8c57cc39 (table default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]) successfully deleted
I20260926 19:58:37.460367 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 8538b46e783f4ffb9a31c12939f954af
I20260926 19:58:37.461995 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet 8538b46e783f4ffb9a31c12939f954af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:37.462370 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet ad42626eb2054e7fb549f1c4d20b4cfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:37.463091 13060 tablet_replica.cc:333] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:37.463835 13060 raft_consensus.cc:2243] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:37.464558 13060 pending_rounds.cc:70] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:37.464781 13060 pending_rounds.cc:77] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694329002549248 op_type: NO_OP noop_request { }
I20260926 19:58:37.465663 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 8538b46e783f4ffb9a31c12939f954af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:37.466574 13032 tablet_replica.cc:333] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:37.467283 13032 raft_consensus.cc:2243] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.467734 13032 raft_consensus.cc:2272] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.470795 13060 raft_consensus.cc:2889] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:37.471262 13060 raft_consensus.cc:2272] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.473955 13060 ts_tablet_manager.cc:1916] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.475467 13012 tablet_bootstrap.cc:492] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:37.477661 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 4e82262ea1c441808e9765aef911c903 (DEFAULT_TABLE table=default.Table_c7b5800561444fd7ad044784765fd2bb [id=68ea017f68a5467c8c1d1c1ea98f4701]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.478724 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e82262ea1c441808e9765aef911c903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.478825 13012 ts_tablet_manager.cc:1403] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.054s user 0.011s sys 0.000s
I20260926 19:58:37.481361 13012 raft_consensus.cc:359] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.489722 13012 raft_consensus.cc:385] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.490069 13012 raft_consensus.cc:740] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
W20260926 19:58:37.490238 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4eca1e5734324f99b9362915bdd00239 consensus state has no leader
I20260926 19:58:37.490859 13012 consensus_queue.cc:260] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.492661 13012 ts_tablet_manager.cc:1434] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
W20260926 19:58:37.498507 12382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.253:37913, user_credentials={real_user=slave}} blocked reactor thread for 86931.5us
I20260926 19:58:37.503376 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 8538b46e783f4ffb9a31c12939f954af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:37.503746 13012 tablet_bootstrap.cc:492] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:37.509387 13012 tablet_bootstrap.cc:654] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.510846 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 6f1dafade40a46649ac2780b8ad3b9ff (DEFAULT_TABLE table=default.Table_45d30c1ab9414653bfb03865e36d16b7 [id=e8e5454a4f9243988f48a9c7d26a7736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.512383 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f1dafade40a46649ac2780b8ad3b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.512988 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 4e82262ea1c441808e9765aef911c903 (DEFAULT_TABLE table=default.Table_c7b5800561444fd7ad044784765fd2bb [id=68ea017f68a5467c8c1d1c1ea98f4701]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.514150 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e82262ea1c441808e9765aef911c903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.522104 13012 tablet_bootstrap.cc:492] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:37.522490 13012 ts_tablet_manager.cc:1403] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.000s
I20260926 19:58:37.525446 13032 ts_tablet_manager.cc:1916] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.528545 13012 raft_consensus.cc:359] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.529181 13012 raft_consensus.cc:385] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.529431 13012 raft_consensus.cc:740] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.530023 13012 consensus_queue.cc:260] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.535650 13092 tablet_bootstrap.cc:492] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:37.536091 13092 ts_tablet_manager.cc:1403] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.289s user 0.015s sys 0.008s
I20260926 19:58:37.538898 13092 raft_consensus.cc:359] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.539465 13060 ts_tablet_manager.cc:1929] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:37.539655 13092 raft_consensus.cc:385] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.539805 13060 log.cc:1199] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/8538b46e783f4ffb9a31c12939f954af
I20260926 19:58:37.540079 13092 raft_consensus.cc:740] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.540563 13060 ts_tablet_manager.cc:1950] T 8538b46e783f4ffb9a31c12939f954af P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:37.540983 13092 consensus_queue.cc:260] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.543493 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 8538b46e783f4ffb9a31c12939f954af (table default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]) successfully deleted
I20260926 19:58:37.543727 13092 ts_tablet_manager.cc:1434] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260926 19:58:37.545470 13012 ts_tablet_manager.cc:1434] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260926 19:58:37.546348 13012 tablet_bootstrap.cc:492] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:37.555734 13012 tablet_bootstrap.cc:654] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.553413 13031 ts_tablet_manager.cc:1929] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.556245 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 6f1dafade40a46649ac2780b8ad3b9ff (DEFAULT_TABLE table=default.Table_45d30c1ab9414653bfb03865e36d16b7 [id=e8e5454a4f9243988f48a9c7d26a7736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.556828 13031 log.cc:1199] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/ec253204a23a44bda812070e8c57cc39
I20260926 19:58:37.557281 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f1dafade40a46649ac2780b8ad3b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.557557 13031 ts_tablet_manager.cc:1950] T ec253204a23a44bda812070e8c57cc39 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:37.575054 13013 tablet_bootstrap.cc:492] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:37.575771 13012 tablet_bootstrap.cc:492] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:37.576166 13012 ts_tablet_manager.cc:1403] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.006s
I20260926 19:58:37.576855 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet ec253204a23a44bda812070e8c57cc39 (table default.Table_f6cb69a2397348c3a00214884a88d013 [id=e4b7dd7ab1144fe9b597ea9240b2b62a]) successfully deleted
I20260926 19:58:37.578454 13012 raft_consensus.cc:359] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.579051 13012 raft_consensus.cc:385] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.579295 13012 raft_consensus.cc:740] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.580191 13031 tablet_replica.cc:333] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:37.579885 13012 consensus_queue.cc:260] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.578679 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 6f1dafade40a46649ac2780b8ad3b9ff (DEFAULT_TABLE table=default.Table_45d30c1ab9414653bfb03865e36d16b7 [id=e8e5454a4f9243988f48a9c7d26a7736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.581821 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f1dafade40a46649ac2780b8ad3b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.582255 12739 tablet_service.cc:1511] Processing CreateTablet for tablet e365004bba0d428cbb1900ad558efe5e (DEFAULT_TABLE table=default.Table_9ac55a637c1e41f0bfca50930bd701c0 [id=47b086c8685e4ac19c14784f1691cae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.583292 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e365004bba0d428cbb1900ad558efe5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.591361 13092 tablet_bootstrap.cc:492] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:37.594671 13012 ts_tablet_manager.cc:1434] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260926 19:58:37.595571 13012 tablet_bootstrap.cc:492] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:37.608157 13013 tablet_bootstrap.cc:654] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.617333 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_cfb0abe6db43480496837ee334f94e25" } new_table_name: "default.Table_0501c3819754464dbfeecf1cc22595ea" modify_external_catalogs: true
I20260926 19:58:37.617842 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a6745a20e46f46c8bda622354e9d558b" } modify_external_catalogs: true
I20260926 19:58:37.617756 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_7b939af633304d308a635b641717d204"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:37.618459 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a6745a20e46f46c8bda622354e9d558b" } modify_external_catalogs: true
I20260926 19:58:37.627379 13012 tablet_bootstrap.cc:654] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:37.629069 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b939af633304d308a635b641717d204 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:37.648299 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet e20905f83b2e4c7fa409fce38ec5f81e
I20260926 19:58:37.650380 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet e20905f83b2e4c7fa409fce38ec5f81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:37.651758 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet e20905f83b2e4c7fa409fce38ec5f81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:37.652570 13060 tablet_replica.cc:333] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:37.653227 13060 raft_consensus.cc:2243] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.653681 13060 raft_consensus.cc:2272] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.656117 13060 ts_tablet_manager.cc:1916] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.672145 12568 tablet_service.cc:1511] Processing CreateTablet for tablet e365004bba0d428cbb1900ad558efe5e (DEFAULT_TABLE table=default.Table_9ac55a637c1e41f0bfca50930bd701c0 [id=47b086c8685e4ac19c14784f1691cae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.673734 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e365004bba0d428cbb1900ad558efe5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.682026 13032 ts_tablet_manager.cc:1929] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260926 19:58:37.686439 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c1822d4c697d4c4985902355fc360dd1 consensus state has no leader
I20260926 19:58:37.688783 13060 ts_tablet_manager.cc:1929] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.689168 13060 log.cc:1199] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/e20905f83b2e4c7fa409fce38ec5f81e
I20260926 19:58:37.690428 13032 log.cc:1199] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/ad42626eb2054e7fb549f1c4d20b4cfc
I20260926 19:58:37.594991 13031 raft_consensus.cc:2243] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.692231 13032 ts_tablet_manager.cc:1950] T ad42626eb2054e7fb549f1c4d20b4cfc P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:37.692307 13031 raft_consensus.cc:2272] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.694232 13031 ts_tablet_manager.cc:1916] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.696388 12741 tablet_service.cc:1511] Processing CreateTablet for tablet c1822d4c697d4c4985902355fc360dd1 (DEFAULT_TABLE table=default.Table_0501c3819754464dbfeecf1cc22595ea [id=b1955ec80c98477c95b3b06cb73f3206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.700129 12565 tablet_service.cc:1511] Processing CreateTablet for tablet c1822d4c697d4c4985902355fc360dd1 (DEFAULT_TABLE table=default.Table_0501c3819754464dbfeecf1cc22595ea [id=b1955ec80c98477c95b3b06cb73f3206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.701196 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1822d4c697d4c4985902355fc360dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.702839 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1822d4c697d4c4985902355fc360dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.705791 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_033bac717553494ebf0ce0b775144fcf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:37.706890 12388 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1b0fc924e4754830b2e296f3835dd9d2" } modify_external_catalogs: true
I20260926 19:58:37.707242 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_7b939af633304d308a635b641717d204" } new_table_name: "default.Table_55fcc26215f84225ba3230f2c882843c" modify_external_catalogs: true
I20260926 19:58:37.707422 12388 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1b0fc924e4754830b2e296f3835dd9d2" } modify_external_catalogs: true
W20260926 19:58:37.708537 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_033bac717553494ebf0ce0b775144fcf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:37.706700 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet ad42626eb2054e7fb549f1c4d20b4cfc (table default.Table_55128b0718b349d18a8f378ecd6d0a24 [id=7606ba867d204bbcb4dc88918435fb1a]) successfully deleted
I20260926 19:58:37.709615 13095 raft_consensus.cc:493] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:37.710076 13095 raft_consensus.cc:515] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.712042 13095 leader_election.cc:290] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:37.713981 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_c46bbc68304f4ebb80a240478a92349d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:37.716073 13032 tablet_replica.cc:333] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:37.716845 13032 raft_consensus.cc:2243] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.717299 13032 raft_consensus.cc:2272] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.719135 13032 ts_tablet_manager.cc:1916] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.709291 13060 ts_tablet_manager.cc:1950] T e20905f83b2e4c7fa409fce38ec5f81e P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:37.717152 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f3de5e9b6641508ba56f9b33e930f2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:37.721900 12595 raft_consensus.cc:2468] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:37.723982 12695 leader_election.cc:304] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:37.724850 13095 raft_consensus.cc:2804] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:37.725167 13095 raft_consensus.cc:493] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:37.725445 13095 raft_consensus.cc:3060] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.730161 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet e20905f83b2e4c7fa409fce38ec5f81e (table default.Table_a6745a20e46f46c8bda622354e9d558b [id=c699605165e847f4bca3d95b659eb989]) successfully deleted
I20260926 19:58:37.732908 13095 raft_consensus.cc:515] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.735052 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet e20905f83b2e4c7fa409fce38ec5f81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:37.737114 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f3de5e9b6641508ba56f9b33e930f2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:37.737658 12595 raft_consensus.cc:3060] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.745851 12901 tablet_service.cc:1511] Processing CreateTablet for tablet e365004bba0d428cbb1900ad558efe5e (DEFAULT_TABLE table=default.Table_9ac55a637c1e41f0bfca50930bd701c0 [id=47b086c8685e4ac19c14784f1691cae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.748014 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e365004bba0d428cbb1900ad558efe5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.737462 13092 tablet_bootstrap.cc:654] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:37.752471 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c46bbc68304f4ebb80a240478a92349d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:37.754118 12900 tablet_service.cc:1511] Processing CreateTablet for tablet c1822d4c697d4c4985902355fc360dd1 (DEFAULT_TABLE table=default.Table_0501c3819754464dbfeecf1cc22595ea [id=b1955ec80c98477c95b3b06cb73f3206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.755869 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1822d4c697d4c4985902355fc360dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.759977 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f3de5e9b6641508ba56f9b33e930f2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:37.753633 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet ad42626eb2054e7fb549f1c4d20b4cfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:36 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:37.762609 13018 raft_consensus.cc:493] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:37.750237 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f3de5e9b6641508ba56f9b33e930f2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:37.763049 13018 raft_consensus.cc:515] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.764854 13018 leader_election.cc:290] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:58:37.768344 12696 leader_election.cc:343] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:37.769496 12696 leader_election.cc:343] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:37.772338 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05a6d163900947689b24489787bc119c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:37.774098 12696 leader_election.cc:343] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:37.775846 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05a6d163900947689b24489787bc119c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:37.776463 12594 raft_consensus.cc:2468] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:37.772636 13032 ts_tablet_manager.cc:1929] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.777391 13032 log.cc:1199] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/8538b46e783f4ffb9a31c12939f954af
I20260926 19:58:37.777829 12695 leader_election.cc:304] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:37.780234 13018 raft_consensus.cc:2804] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:37.780628 13018 raft_consensus.cc:493] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:37.780859 13090 raft_consensus.cc:493] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:37.781023 13018 raft_consensus.cc:3060] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.781332 13090 raft_consensus.cc:515] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.783308 13090 leader_election.cc:290] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:37.784834 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e20905f83b2e4c7fa409fce38ec5f81e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:37.786247 12696 leader_election.cc:343] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:37.788297 13018 raft_consensus.cc:515] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:58:37.788707 12695 leader_election.cc:343] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: e20905f83b2e4c7fa409fce38ec5f81e
I20260926 19:58:37.789222 12695 leader_election.cc:304] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:37.789964 13090 raft_consensus.cc:2749] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:37.790692 13018 leader_election.cc:290] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:37.791595 13032 ts_tablet_manager.cc:1950] T 8538b46e783f4ffb9a31c12939f954af P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:37.792162 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05a6d163900947689b24489787bc119c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:37.793736 12696 leader_election.cc:343] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:37.795673 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05a6d163900947689b24489787bc119c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:37.796253 12594 raft_consensus.cc:3060] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:37.797168 13092 tablet_bootstrap.cc:492] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:37.797823 13092 ts_tablet_manager.cc:1403] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.207s user 0.011s sys 0.001s
I20260926 19:58:37.798138 13032 tablet_replica.cc:333] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:37.798836 13032 raft_consensus.cc:2243] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.799270 13032 raft_consensus.cc:2272] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.794317 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 8538b46e783f4ffb9a31c12939f954af (table default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]) successfully deleted
W20260926 19:58:37.802253 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1283b93b610a4857910aefe8b2737495 has no consensus state
I20260926 19:58:37.808046 13032 ts_tablet_manager.cc:1916] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.745297 12595 raft_consensus.cc:2468] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:37.821154 12695 leader_election.cc:304] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:37.821928 13018 raft_consensus.cc:2804] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:37.822202 13095 leader_election.cc:290] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:37.822331 13018 raft_consensus.cc:697] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:37.823048 13018 consensus_queue.cc:237] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.823908 12594 raft_consensus.cc:2468] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:37.825006 12695 leader_election.cc:304] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:37.825723 13090 raft_consensus.cc:2804] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:37.826331 13090 raft_consensus.cc:697] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:37.827092 13090 consensus_queue.cc:237] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.847138 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 4e706596959e400398bf2213653f06c9
I20260926 19:58:37.850006 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e706596959e400398bf2213653f06c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:37.854256 13032 ts_tablet_manager.cc:1929] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.854678 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9f3804455bdf4e8f81dcbf449bdb9c03"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:37.855074 13032 log.cc:1199] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/e20905f83b2e4c7fa409fce38ec5f81e
I20260926 19:58:37.855719 13032 ts_tablet_manager.cc:1950] T e20905f83b2e4c7fa409fce38ec5f81e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
W20260926 19:58:37.856354 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f3804455bdf4e8f81dcbf449bdb9c03 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:37.853554 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e706596959e400398bf2213653f06c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:37.858229 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet e20905f83b2e4c7fa409fce38ec5f81e (table default.Table_a6745a20e46f46c8bda622354e9d558b [id=c699605165e847f4bca3d95b659eb989]) successfully deleted
I20260926 19:58:37.861898 13032 tablet_replica.cc:333] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:37.863057 12388 catalog_manager.cc:5838] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:37.860448 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_55fcc26215f84225ba3230f2c882843c" } modify_external_catalogs: true
I20260926 19:58:37.869021 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_55fcc26215f84225ba3230f2c882843c" } modify_external_catalogs: true
I20260926 19:58:37.869172 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_353f84cf17724709af77fc8d316ca0c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:37.814137 13092 raft_consensus.cc:359] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.870052 13092 raft_consensus.cc:385] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:58:37.870827 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_353f84cf17724709af77fc8d316ca0c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:37.870435 13092 raft_consensus.cc:740] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.871928 13092 consensus_queue.cc:260] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.876060 12564 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e706596959e400398bf2213653f06c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:37.881489 13032 raft_consensus.cc:2243] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:37.882972 13032 pending_rounds.cc:70] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Trying to abort 2 pending ops.
I20260926 19:58:37.883402 12923 raft_consensus.cc:1275] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Refusing update from remote peer 3ae87963ed5e4d92a896bf4a73af4da9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:37.883253 13032 pending_rounds.cc:77] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333694331678801920 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "4e706596959e400398bf2213653f06c9" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 3 new_table_name: "default.Table_1b0fc924e4754830b2e296f3835dd9d2" dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" new_extra_config { } }
W20260926 19:58:37.884157 13032 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38310: Aborted: Op aborted
I20260926 19:58:37.884927 13032 pending_rounds.cc:77] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694330251382784 op_type: NO_OP noop_request { }
I20260926 19:58:37.885288 13032 raft_consensus.cc:2889] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:37.885624 13032 raft_consensus.cc:2272] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.886268 13092 ts_tablet_manager.cc:1434] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.088s user 0.007s sys 0.000s
I20260926 19:58:37.887806 13032 ts_tablet_manager.cc:1916] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.892719 13013 tablet_bootstrap.cc:492] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:37.893126 13013 ts_tablet_manager.cc:1403] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.318s user 0.005s sys 0.009s
I20260926 19:58:37.893433 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_26612903c7e7487bb95439ebde7b204e" } new_table_name: "default.Table_3ccdb238d9bb45689fc56322b90e12c3" modify_external_catalogs: true
I20260926 19:58:37.894829 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 00767b01d2ad4ca5b6cfcc566739f61d (DEFAULT_TABLE table=default.Table_d2d87877a7c4476aa7e41623c09ed4df [id=d4475cbaf3a74d19818f03cdb5ca6537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.895901 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00767b01d2ad4ca5b6cfcc566739f61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.897639 13012 tablet_bootstrap.cc:492] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:37.898034 13012 ts_tablet_manager.cc:1403] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.303s user 0.036s sys 0.040s
I20260926 19:58:37.897915 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 00767b01d2ad4ca5b6cfcc566739f61d (DEFAULT_TABLE table=default.Table_d2d87877a7c4476aa7e41623c09ed4df [id=d4475cbaf3a74d19818f03cdb5ca6537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.898983 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00767b01d2ad4ca5b6cfcc566739f61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.900388 13012 raft_consensus.cc:359] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.901010 13012 raft_consensus.cc:385] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.901253 13012 raft_consensus.cc:740] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.901868 13012 consensus_queue.cc:260] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.902452 13013 raft_consensus.cc:359] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.903081 13013 raft_consensus.cc:385] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.903330 13013 raft_consensus.cc:740] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.903887 13013 consensus_queue.cc:260] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.909206 13060 tablet_replica.cc:333] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:37.909873 13060 raft_consensus.cc:2243] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:37.911312 13012 ts_tablet_manager.cc:1434] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260926 19:58:37.919622 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 4eca1e5734324f99b9362915bdd00239 (DEFAULT_TABLE table=default.Table_dc1bdf9e02c44b0bb32ccf1e18872c04 [id=4692b08d69b344cdaf9226c62886da70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.920714 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eca1e5734324f99b9362915bdd00239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.922029 13032 ts_tablet_manager.cc:1929] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:37.922360 13032 log.cc:1199] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/4e706596959e400398bf2213653f06c9
I20260926 19:58:37.923002 13032 ts_tablet_manager.cc:1950] T 4e706596959e400398bf2213653f06c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:37.926826 12390 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 1283b93b610a4857910aefe8b2737495
I20260926 19:58:37.930919 13012 tablet_bootstrap.cc:492] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:37.936621 13012 tablet_bootstrap.cc:654] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.939893 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 4eca1e5734324f99b9362915bdd00239 (DEFAULT_TABLE table=default.Table_dc1bdf9e02c44b0bb32ccf1e18872c04 [id=4692b08d69b344cdaf9226c62886da70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.940984 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eca1e5734324f99b9362915bdd00239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.925967 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 00767b01d2ad4ca5b6cfcc566739f61d (DEFAULT_TABLE table=default.Table_d2d87877a7c4476aa7e41623c09ed4df [id=d4475cbaf3a74d19818f03cdb5ca6537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:37.942242 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00767b01d2ad4ca5b6cfcc566739f61d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:37.946353 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 4e706596959e400398bf2213653f06c9 (table default.Table_1b0fc924e4754830b2e296f3835dd9d2 [id=e2cf63902d2c4d28815fca606eef0274]) successfully deleted
I20260926 19:58:37.946518 13060 raft_consensus.cc:2272] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:37.948375 13060 ts_tablet_manager.cc:1916] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:37.951408 13012 tablet_bootstrap.cc:492] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:37.951826 13012 ts_tablet_manager.cc:1403] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.001s
W20260926 19:58:37.952674 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 82f3de5e9b6641508ba56f9b33e930f2 consensus state has no leader
I20260926 19:58:37.955117 13012 raft_consensus.cc:359] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.955771 13012 raft_consensus.cc:385] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:37.956027 13012 raft_consensus.cc:740] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:37.957479 13012 consensus_queue.cc:260] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:37.962173 13092 tablet_bootstrap.cc:492] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:37.963193 13012 ts_tablet_manager.cc:1434] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.011s user 0.002s sys 0.005s
I20260926 19:58:37.964361 13012 tablet_bootstrap.cc:492] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:37.964781 13013 ts_tablet_manager.cc:1434] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.071s user 0.005s sys 0.000s
I20260926 19:58:37.966073 13013 tablet_bootstrap.cc:492] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:37.969595 13012 tablet_bootstrap.cc:654] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:37.992216 13031 ts_tablet_manager.cc:1929] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:37.993003 13031 log.cc:1199] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/8538b46e783f4ffb9a31c12939f954af
I20260926 19:58:38.005823 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_45d30c1ab9414653bfb03865e36d16b7" } new_table_name: "default.Table_b3e35b31063d4ba7a63fc5af3f5dd752" modify_external_catalogs: true
I20260926 19:58:38.022101 13031 ts_tablet_manager.cc:1950] T 8538b46e783f4ffb9a31c12939f954af P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
W20260926 19:58:38.022507 12696 consensus_peers.cc:597] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:38.007007 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 4eca1e5734324f99b9362915bdd00239 (DEFAULT_TABLE table=default.Table_dc1bdf9e02c44b0bb32ccf1e18872c04 [id=4692b08d69b344cdaf9226c62886da70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.024142 12594 raft_consensus.cc:1275] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Refusing update from remote peer 3ae87963ed5e4d92a896bf4a73af4da9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:38.024971 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eca1e5734324f99b9362915bdd00239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.025748 13095 consensus_queue.cc:1048] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:38.033557 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 8538b46e783f4ffb9a31c12939f954af (table default.Table_9729e85fef9b43b7bc6f5a034c04e20e [id=6fa0d9a0b43d4b459093b84c44dca3b2]) successfully deleted
I20260926 19:58:38.035900 13092 tablet_bootstrap.cc:654] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.040997 13060 ts_tablet_manager.cc:1929] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:38.041350 13060 log.cc:1199] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/4e706596959e400398bf2213653f06c9
I20260926 19:58:38.042112 13060 ts_tablet_manager.cc:1950] T 4e706596959e400398bf2213653f06c9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:38.049953 13031 tablet_replica.cc:333] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:38.050820 13031 raft_consensus.cc:2243] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.052197 13031 raft_consensus.cc:2272] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:38.055800 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f1dafade40a46649ac2780b8ad3b9ff consensus state has no leader
I20260926 19:58:38.060474 13031 ts_tablet_manager.cc:1916] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.064771 12391 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_68d42bf187a146958c5daf317378dafd" } modify_external_catalogs: true
W20260926 19:58:38.064924 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet e20905f83b2e4c7fa409fce38ec5f81e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e20905f83b2e4c7fa409fce38ec5f81e already in progress: creating tablet
I20260926 19:58:38.065315 12391 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_68d42bf187a146958c5daf317378dafd" } modify_external_catalogs: true
I20260926 19:58:38.065965 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_56393c731f9e44fe823755a7e91ccc3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:38.067833 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56393c731f9e44fe823755a7e91ccc3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:38.070103 12382 consensus_peers.cc:597] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 -> Peer db2490adb6e84090b780f52470e580ab (127.11.233.253:37913): Couldn't send request to peer db2490adb6e84090b780f52470e580ab. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260926 19:58:38.085561 13119 tablet.cc:1751] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:38.095216 13013 tablet_bootstrap.cc:654] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.096166 13092 tablet_bootstrap.cc:492] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:38.096570 13092 ts_tablet_manager.cc:1403] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.135s user 0.008s sys 0.004s
W20260926 19:58:38.102007 12382 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 6f1dafade40a46649ac2780b8ad3b9ff (table default.Table_b3e35b31063d4ba7a63fc5af3f5dd752 [id=e8e5454a4f9243988f48a9c7d26a7736], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6f1dafade40a46649ac2780b8ad3b9ff consensus state has no leader
I20260926 19:58:38.099752 13092 raft_consensus.cc:359] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.105038 13092 raft_consensus.cc:385] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.105314 13092 raft_consensus.cc:740] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.110625 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 4e706596959e400398bf2213653f06c9 (table default.Table_1b0fc924e4754830b2e296f3835dd9d2 [id=e2cf63902d2c4d28815fca606eef0274]) successfully deleted
I20260926 19:58:38.111864 13013 tablet_bootstrap.cc:492] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:38.112241 13013 ts_tablet_manager.cc:1403] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.146s user 0.013s sys 0.000s
I20260926 19:58:38.115010 13013 raft_consensus.cc:359] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.115748 13013 raft_consensus.cc:385] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.115583 13122 tablet.cc:1751] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:38.116854 13012 tablet_bootstrap.cc:492] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.117295 13012 ts_tablet_manager.cc:1403] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.153s user 0.032s sys 0.018s
I20260926 19:58:38.121251 13092 consensus_queue.cc:260] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.116039 13013 raft_consensus.cc:740] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.121722 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_bcd1e36898864327831252b825b13da6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.122714 13013 consensus_queue.cc:260] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:58:38.123478 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcd1e36898864327831252b825b13da6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.124923 13012 raft_consensus.cc:359] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.127305 13012 raft_consensus.cc:385] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.127615 13012 raft_consensus.cc:740] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.128698 13012 consensus_queue.cc:260] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.134024 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9ac55a637c1e41f0bfca50930bd701c0" } modify_external_catalogs: true
I20260926 19:58:38.134559 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9ac55a637c1e41f0bfca50930bd701c0" } modify_external_catalogs: true
I20260926 19:58:38.136960 13013 ts_tablet_manager.cc:1434] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.024s user 0.003s sys 0.002s
I20260926 19:58:38.140853 13013 tablet_bootstrap.cc:492] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:38.140838 13092 ts_tablet_manager.cc:1434] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.044s user 0.009s sys 0.000s
I20260926 19:58:38.146217 13013 tablet_bootstrap.cc:654] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.151870 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet e20905f83b2e4c7fa409fce38ec5f81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:37 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:38.163000 12388 catalog_manager.cc:5838] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:38.164263 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 8c683dc1272f4290a56f7fc7010ba038 (DEFAULT_TABLE table=default.Table_c46bbc68304f4ebb80a240478a92349d [id=f1d58054ff284e03af21a1f7e857732e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.165311 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c683dc1272f4290a56f7fc7010ba038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.169926 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 8c683dc1272f4290a56f7fc7010ba038 (DEFAULT_TABLE table=default.Table_c46bbc68304f4ebb80a240478a92349d [id=f1d58054ff284e03af21a1f7e857732e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.170977 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c683dc1272f4290a56f7fc7010ba038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.170779 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 8c683dc1272f4290a56f7fc7010ba038 (DEFAULT_TABLE table=default.Table_c46bbc68304f4ebb80a240478a92349d [id=f1d58054ff284e03af21a1f7e857732e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.171761 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c683dc1272f4290a56f7fc7010ba038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.173764 13092 tablet_bootstrap.cc:492] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:38.182753 12904 tablet_service.cc:1511] Processing CreateTablet for tablet b46a5297ef854afd8ef783bf0692135e (DEFAULT_TABLE table=default.Table_033bac717553494ebf0ce0b775144fcf [id=a774283438c64c668a5b2e9e4d62ed99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.183924 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46a5297ef854afd8ef783bf0692135e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.183665 12565 tablet_service.cc:1511] Processing CreateTablet for tablet b46a5297ef854afd8ef783bf0692135e (DEFAULT_TABLE table=default.Table_033bac717553494ebf0ce0b775144fcf [id=a774283438c64c668a5b2e9e4d62ed99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.184684 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46a5297ef854afd8ef783bf0692135e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.186329 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_cba4802138f54b36a76988eddd698d94" } new_table_name: "default.Table_ed4e65d4240d48f88a62a3b5e15174bd" modify_external_catalogs: true
I20260926 19:58:38.191534 13031 ts_tablet_manager.cc:1929] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:38.193238 12741 tablet_service.cc:1511] Processing CreateTablet for tablet b46a5297ef854afd8ef783bf0692135e (DEFAULT_TABLE table=default.Table_033bac717553494ebf0ce0b775144fcf [id=a774283438c64c668a5b2e9e4d62ed99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.196215 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b46a5297ef854afd8ef783bf0692135e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.201296 13012 ts_tablet_manager.cc:1434] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.084s user 0.005s sys 0.001s
I20260926 19:58:38.202183 13012 tablet_bootstrap.cc:492] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.208245 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b1794633093d4c15bfaa7f160828ff99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.209002 12739 tablet_service.cc:1511] Processing CreateTablet for tablet d38d785b63734ddd9f2e4f47ef55c7c9 (DEFAULT_TABLE table=default.Table_cba4802138f54b36a76988eddd698d94 [id=244a65d5edb64b3b8b8ca011dfbc9e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.210053 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38d785b63734ddd9f2e4f47ef55c7c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:38.211515 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1794633093d4c15bfaa7f160828ff99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.211953 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_724fda2a18204a24979779d6e0aafb81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:38.213781 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_724fda2a18204a24979779d6e0aafb81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.216943 12568 tablet_service.cc:1511] Processing CreateTablet for tablet d38d785b63734ddd9f2e4f47ef55c7c9 (DEFAULT_TABLE table=default.Table_cba4802138f54b36a76988eddd698d94 [id=244a65d5edb64b3b8b8ca011dfbc9e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.217950 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38d785b63734ddd9f2e4f47ef55c7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.195266 12903 tablet_service.cc:1511] Processing CreateTablet for tablet d38d785b63734ddd9f2e4f47ef55c7c9 (DEFAULT_TABLE table=default.Table_cba4802138f54b36a76988eddd698d94 [id=244a65d5edb64b3b8b8ca011dfbc9e24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.222328 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38d785b63734ddd9f2e4f47ef55c7c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.225965 13048 raft_consensus.cc:493] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:38.226418 13048 raft_consensus.cc:515] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.228365 13048 leader_election.cc:290] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:38.229436 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05a6d163900947689b24489787bc119c" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:38.229576 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05a6d163900947689b24489787bc119c" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:38.230198 12595 raft_consensus.cc:2393] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:38.231506 12857 leader_election.cc:304] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:38.232611 13048 raft_consensus.cc:3060] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:38.240049 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 99cf26cf4838454c98aa8337341ffd06 (DEFAULT_TABLE table=default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.241364 13039 raft_consensus.cc:493] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:38.241572 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99cf26cf4838454c98aa8337341ffd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.241827 13039 raft_consensus.cc:515] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.244194 13039 leader_election.cc:290] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:38.246163 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e82262ea1c441808e9765aef911c903" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:38.246721 12760 raft_consensus.cc:2468] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:38.248914 12519 leader_election.cc:304] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:38.250861 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e82262ea1c441808e9765aef911c903" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:38.252413 12521 leader_election.cc:343] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:38.253273 13039 raft_consensus.cc:2804] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:38.253687 13039 raft_consensus.cc:493] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:38.254487 13039 raft_consensus.cc:3060] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:38.260015 13048 raft_consensus.cc:2749] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:38.260689 13031 log.cc:1199] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/ad42626eb2054e7fb549f1c4d20b4cfc
I20260926 19:58:38.261323 13031 ts_tablet_manager.cc:1950] T ad42626eb2054e7fb549f1c4d20b4cfc P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:38.263934 13031 tablet_replica.cc:333] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:38.264226 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet ad42626eb2054e7fb549f1c4d20b4cfc (table default.Table_55128b0718b349d18a8f378ecd6d0a24 [id=7606ba867d204bbcb4dc88918435fb1a]) successfully deleted
I20260926 19:58:38.264534 13031 raft_consensus.cc:2243] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.264931 13031 raft_consensus.cc:2272] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:38.275282 12395 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d38d785b63734ddd9f2e4f47ef55c7c9 consensus state has no leader
I20260926 19:58:38.220422 13092 tablet_bootstrap.cc:654] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.300876 12394 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_ee0eba1e9d7548799bca25f3815c5ad3" } new_table_name: "default.Table_14c589443e094e22bc1c798043fd6d04" modify_external_catalogs: true
I20260926 19:58:38.303390 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 99cf26cf4838454c98aa8337341ffd06 (DEFAULT_TABLE table=default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.304370 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99cf26cf4838454c98aa8337341ffd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.305228 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 99cf26cf4838454c98aa8337341ffd06 (DEFAULT_TABLE table=default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.306263 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99cf26cf4838454c98aa8337341ffd06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.309863 12387 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 6f1dafade40a46649ac2780b8ad3b9ff (table default.Table_b3e35b31063d4ba7a63fc5af3f5dd752 [id=e8e5454a4f9243988f48a9c7d26a7736]). Expected version 1 got 0
I20260926 19:58:38.317274 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f6554f3a9eb54288a5b4b0a316bfbbf6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.317727 12564 tablet_service.cc:1511] Processing CreateTablet for tablet ef062e1775bd4a3c859124eaaba81399 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.318555 13031 ts_tablet_manager.cc:1916] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.318771 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef062e1775bd4a3c859124eaaba81399. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:38.319188 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6554f3a9eb54288a5b4b0a316bfbbf6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.332553 12391 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 05a6d163900947689b24489787bc119c
I20260926 19:58:38.335841 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 05a6d163900947689b24489787bc119c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:38.336630 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 05a6d163900947689b24489787bc119c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:38.337365 13032 tablet_replica.cc:333] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:38.338080 13032 raft_consensus.cc:2243] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:38.338699 13032 pending_rounds.cc:70] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Trying to abort 1 pending ops.
I20260926 19:58:38.338905 13032 pending_rounds.cc:77] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694332228927488 op_type: NO_OP noop_request { }
I20260926 19:58:38.339239 13032 raft_consensus.cc:2889] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:38.339545 13032 raft_consensus.cc:2272] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:38.341949 13032 ts_tablet_manager.cc:1916] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.347790 13039 raft_consensus.cc:515] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.352103 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 05a6d163900947689b24489787bc119c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:38.352797 13060 tablet_replica.cc:333] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:38.353995 13060 raft_consensus.cc:2243] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.354581 13060 raft_consensus.cc:2272] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:38.356645 13060 ts_tablet_manager.cc:1916] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.330420 12903 tablet_service.cc:1511] Processing CreateTablet for tablet ef062e1775bd4a3c859124eaaba81399 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.358011 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef062e1775bd4a3c859124eaaba81399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.367178 13092 tablet_bootstrap.cc:492] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:38.367269 13012 tablet_bootstrap.cc:654] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.367702 13092 ts_tablet_manager.cc:1403] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.194s user 0.008s sys 0.004s
I20260926 19:58:38.370497 13092 raft_consensus.cc:359] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.371206 13092 raft_consensus.cc:385] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.371109 13013 tablet_bootstrap.cc:492] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:38.371500 13092 raft_consensus.cc:740] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.371933 13013 ts_tablet_manager.cc:1403] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.234s user 0.027s sys 0.023s
I20260926 19:58:38.372390 13092 consensus_queue.cc:260] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.396173 12739 tablet_service.cc:1511] Processing CreateTablet for tablet ef062e1775bd4a3c859124eaaba81399 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.396657 13060 ts_tablet_manager.cc:1929] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:38.397030 13060 log.cc:1199] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/05a6d163900947689b24489787bc119c
I20260926 19:58:38.397331 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet e365004bba0d428cbb1900ad558efe5e
I20260926 19:58:38.397639 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef062e1775bd4a3c859124eaaba81399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.397701 13060 ts_tablet_manager.cc:1950] T 05a6d163900947689b24489787bc119c P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:38.400328 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet e365004bba0d428cbb1900ad558efe5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:38.401580 12387 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_3ccdb238d9bb45689fc56322b90e12c3 [id=f53ec6e521114c0c9cc60e7627db131c] alter complete (version 1)
I20260926 19:58:38.403246 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet e365004bba0d428cbb1900ad558efe5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:38.404803 13092 ts_tablet_manager.cc:1434] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20260926 19:58:38.410493 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.253:37913
--webserver_interface=127.11.233.253
--webserver_port=36963
--builtin_ntp_servers=127.11.233.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 19:58:38.421793 13031 ts_tablet_manager.cc:1929] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:38.422153 13031 log.cc:1199] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/4e706596959e400398bf2213653f06c9
I20260926 19:58:38.424422 13031 ts_tablet_manager.cc:1950] T 4e706596959e400398bf2213653f06c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:38.428896 13013 raft_consensus.cc:359] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:38.429586 13013 raft_consensus.cc:385] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.429854 13013 raft_consensus.cc:740] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.430403 13013 consensus_queue.cc:260] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:38.436292 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 9acbd4e5aa23433d860d399d78485d0d (DEFAULT_TABLE table=default.Table_170a499a114147fc9650b978b327f7e8 [id=6890560cf0104034a9eb9431a38c99ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.437880 13032 ts_tablet_manager.cc:1929] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:38.438232 13032 log.cc:1199] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/05a6d163900947689b24489787bc119c
I20260926 19:58:38.438911 13032 ts_tablet_manager.cc:1950] T 05a6d163900947689b24489787bc119c P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:38.444519 13128 raft_consensus.cc:493] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:38.444976 13128 raft_consensus.cc:515] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:38.445747 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9acbd4e5aa23433d860d399d78485d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.447453 13128 leader_election.cc:290] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:38.448259 13092 tablet_bootstrap.cc:492] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:38.448166 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eca1e5734324f99b9362915bdd00239" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:38.449623 13013 ts_tablet_manager.cc:1434] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.077s user 0.009s sys 0.000s
W20260926 19:58:38.449733 12519 leader_election.cc:343] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:38.450742 13013 tablet_bootstrap.cc:492] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:38.455399 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 4e706596959e400398bf2213653f06c9 (table default.Table_1b0fc924e4754830b2e296f3835dd9d2 [id=e2cf63902d2c4d28815fca606eef0274]) successfully deleted
I20260926 19:58:38.455700 13031 tablet_replica.cc:333] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:38.462011 13039 leader_election.cc:290] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:38.462492 13127 raft_consensus.cc:493] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:38.462795 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e82262ea1c441808e9765aef911c903" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:38.462992 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eca1e5734324f99b9362915bdd00239" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:38.463315 12760 raft_consensus.cc:3060] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:38.463783 13032 tablet_replica.cc:333] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:38.464418 13032 raft_consensus.cc:2243] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.462906 13127 raft_consensus.cc:515] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.456321 13031 raft_consensus.cc:2243] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.465265 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e82262ea1c441808e9765aef911c903" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:38.466418 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 05a6d163900947689b24489787bc119c (table default.Table_68d42bf187a146958c5daf317378dafd [id=37adf5d8d6c1411b8e6028df63dcd641]) successfully deleted
W20260926 19:58:38.466626 12521 leader_election.cc:343] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:38.467072 12521 leader_election.cc:304] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:38.465377 13031 raft_consensus.cc:2272] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:38.468367 12521 leader_election.cc:343] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:38.469894 13128 raft_consensus.cc:2749] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:38.474555 13092 tablet_bootstrap.cc:654] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.479067 13013 tablet_bootstrap.cc:654] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.479944 12392 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 4eca1e5734324f99b9362915bdd00239 (table default.Table_dc1bdf9e02c44b0bb32ccf1e18872c04 [id=4692b08d69b344cdaf9226c62886da70]). Expected version 2 got 0
I20260926 19:58:38.481671 12394 master_service.cc:960] ReplaceTablet: received request to replace tablet ef062e1775bd4a3c859124eaaba81399 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:38.482271 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_be1467ad5f7e4a92b7f91320650f32bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.483072 12760 raft_consensus.cc:2468] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:38.483270 13127 leader_election.cc:290] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:38.483116 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1dafade40a46649ac2780b8ad3b9ff" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:38.483767 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be1467ad5f7e4a92b7f91320650f32bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.483767 12761 raft_consensus.cc:2468] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:38.484414 12519 leader_election.cc:304] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:38.486702 12519 leader_election.cc:304] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:38.487164 13031 ts_tablet_manager.cc:1916] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.487668 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1dafade40a46649ac2780b8ad3b9ff" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:38.489324 13048 raft_consensus.cc:493] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:38.489548 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_dc1bdf9e02c44b0bb32ccf1e18872c04" } new_table_name: "default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95" modify_external_catalogs: true
I20260926 19:58:38.489918 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 05a6d163900947689b24489787bc119c (table default.Table_68d42bf187a146958c5daf317378dafd [id=37adf5d8d6c1411b8e6028df63dcd641]) successfully deleted
I20260926 19:58:38.489782 13048 raft_consensus.cc:515] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.491132 13039 raft_consensus.cc:2804] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:38.491699 13128 raft_consensus.cc:2804] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:38.491704 13048 leader_election.cc:290] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:38.492605 12388 master_service.cc:960] ReplaceTablet: received request to replace tablet 4eca1e5734324f99b9362915bdd00239 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:38.492000 13128 raft_consensus.cc:493] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:38.493342 13128 raft_consensus.cc:3060] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:38.494670 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f3de5e9b6641508ba56f9b33e930f2" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:38.494736 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82f3de5e9b6641508ba56f9b33e930f2" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:38.496321 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_72fe02818135414ca8fb8033ebda39c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.496682 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet e365004bba0d428cbb1900ad558efe5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:38.497121 12857 leader_election.cc:304] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:38.497562 13039 raft_consensus.cc:697] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
W20260926 19:58:38.498490 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72fe02818135414ca8fb8033ebda39c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.498601 13048 raft_consensus.cc:2749] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:38.498420 13039 consensus_queue.cc:237] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.503465 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 9acbd4e5aa23433d860d399d78485d0d (DEFAULT_TABLE table=default.Table_170a499a114147fc9650b978b327f7e8 [id=6890560cf0104034a9eb9431a38c99ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.506891 12390 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 6f1dafade40a46649ac2780b8ad3b9ff (table default.Table_b3e35b31063d4ba7a63fc5af3f5dd752 [id=e8e5454a4f9243988f48a9c7d26a7736]). Expected version 1 got 0
I20260926 19:58:38.510809 12391 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 82f3de5e9b6641508ba56f9b33e930f2 (table default.Table_3ccdb238d9bb45689fc56322b90e12c3 [id=f53ec6e521114c0c9cc60e7627db131c]). Expected version 1 got 0
W20260926 19:58:38.519918 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet e365004bba0d428cbb1900ad558efe5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e365004bba0d428cbb1900ad558efe5e already in progress: creating tablet
I20260926 19:58:38.520195 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 9acbd4e5aa23433d860d399d78485d0d (DEFAULT_TABLE table=default.Table_170a499a114147fc9650b978b327f7e8 [id=6890560cf0104034a9eb9431a38c99ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.521142 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9acbd4e5aa23433d860d399d78485d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.524299 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9acbd4e5aa23433d860d399d78485d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:38.515647 13012 tablet_bootstrap.cc:492] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.529865 13012 ts_tablet_manager.cc:1403] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.328s user 0.008s sys 0.003s
I20260926 19:58:38.532251 13012 raft_consensus.cc:359] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.532925 13012 raft_consensus.cc:385] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.533179 13012 raft_consensus.cc:740] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.533828 13012 consensus_queue.cc:260] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.535550 13012 ts_tablet_manager.cc:1434] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 19:58:38.542518 13012 tablet_bootstrap.cc:492] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.550602 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3ccdb238d9bb45689fc56322b90e12c3" } modify_external_catalogs: true
I20260926 19:58:38.569748 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_1ef063c29bb54a258c2650cf73e9abd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.556012 13128 raft_consensus.cc:515] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.576217 13128 leader_election.cc:290] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:38.578956 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet ef062e1775bd4a3c859124eaaba81399
W20260926 19:58:38.580619 12521 leader_election.cc:343] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:38.580956 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet ef062e1775bd4a3c859124eaaba81399 with delete_type TABLET_DATA_DELETED (replaced by tablet da8875d46f7c46819ee67e650b098c27) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:38.581539 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3ccdb238d9bb45689fc56322b90e12c3" } modify_external_catalogs: true
I20260926 19:58:38.581807 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet ef062e1775bd4a3c859124eaaba81399 with delete_type TABLET_DATA_DELETED (replaced by tablet da8875d46f7c46819ee67e650b098c27) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:38.582643 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1dafade40a46649ac2780b8ad3b9ff" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:38.583362 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet ef062e1775bd4a3c859124eaaba81399 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef062e1775bd4a3c859124eaaba81399 already in progress: creating tablet
I20260926 19:58:38.583690 12394 catalog_manager.cc:6934] ReplaceTablet: tablet ef062e1775bd4a3c859124eaaba81399 deleted and replaced by tablet da8875d46f7c46819ee67e650b098c27
I20260926 19:58:38.464810 13032 raft_consensus.cc:2272] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:38.584194 12521 leader_election.cc:343] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:38.584654 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet ef062e1775bd4a3c859124eaaba81399 with delete_type TABLET_DATA_DELETED (replaced by tablet da8875d46f7c46819ee67e650b098c27) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:38.585793 13032 ts_tablet_manager.cc:1916] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.586459 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f1dafade40a46649ac2780b8ad3b9ff" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:38.587162 12760 raft_consensus.cc:3060] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:38.589568 13012 tablet_bootstrap.cc:654] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.594985 12389 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_0501c3819754464dbfeecf1cc22595ea [id=b1955ec80c98477c95b3b06cb73f3206]). Expected version 1 got 0
I20260926 19:58:38.604640 12760 raft_consensus.cc:2468] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:38.605825 12519 leader_election.cc:304] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:38.607708 13127 raft_consensus.cc:2804] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:38.608410 13127 raft_consensus.cc:697] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:38.609175 13127 consensus_queue.cc:237] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:58:38.613562 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ef063c29bb54a258c2650cf73e9abd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.614749 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b1794633093d4c15bfaa7f160828ff99" } modify_external_catalogs: true
I20260926 19:58:38.620471 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_bcd1e36898864327831252b825b13da6" } new_table_name: "default.Table_bffd1e391c2f49089149f97d1f0be4e9" modify_external_catalogs: true
W20260926 19:58:38.624696 12387 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4eca1e5734324f99b9362915bdd00239 consensus state has no leader
I20260926 19:58:38.627933 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9ef822f7aa5c4261accca0ae1a201026"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.629873 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 4eca1e5734324f99b9362915bdd00239
W20260926 19:58:38.629762 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ef822f7aa5c4261accca0ae1a201026 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.631781 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet 4eca1e5734324f99b9362915bdd00239 with delete_type TABLET_DATA_DELETED (replaced by tablet b19f42511ad14d8a88fc0eaefb82e335) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:38.629185 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_0501c3819754464dbfeecf1cc22595ea [id=b1955ec80c98477c95b3b06cb73f3206]). Expected version 1 got 0
I20260926 19:58:38.641180 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b1794633093d4c15bfaa7f160828ff99" } modify_external_catalogs: true
I20260926 19:58:38.644884 12388 catalog_manager.cc:6934] ReplaceTablet: tablet 4eca1e5734324f99b9362915bdd00239 deleted and replaced by tablet b19f42511ad14d8a88fc0eaefb82e335
I20260926 19:58:38.647060 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 4eca1e5734324f99b9362915bdd00239 with delete_type TABLET_DATA_DELETED (replaced by tablet b19f42511ad14d8a88fc0eaefb82e335) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:38.648353 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 4eca1e5734324f99b9362915bdd00239 with delete_type TABLET_DATA_DELETED (replaced by tablet b19f42511ad14d8a88fc0eaefb82e335) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:38.649011 13060 tablet_replica.cc:333] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:38.649801 13060 raft_consensus.cc:2243] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.650230 13060 raft_consensus.cc:2272] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:38.652653 13031 ts_tablet_manager.cc:1929] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:38.652842 13060 ts_tablet_manager.cc:1916] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.659605 13122 tablet.cc:1751] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:38.661314 13012 tablet_bootstrap.cc:492] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.661720 13012 ts_tablet_manager.cc:1403] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.120s user 0.007s sys 0.012s
I20260926 19:58:38.664629 13012 raft_consensus.cc:359] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.665323 13012 raft_consensus.cc:385] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.665661 13012 raft_consensus.cc:740] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.666327 13012 consensus_queue.cc:260] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.667013 13013 tablet_bootstrap.cc:492] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:38.667402 13013 ts_tablet_manager.cc:1403] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.217s user 0.016s sys 0.005s
I20260926 19:58:38.668161 13012 ts_tablet_manager.cc:1434] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260926 19:58:38.668954 13012 tablet_bootstrap.cc:492] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.675741 13012 tablet_bootstrap.cc:654] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.670596 13013 raft_consensus.cc:359] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.678102 13013 raft_consensus.cc:385] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.678447 13013 raft_consensus.cc:740] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.679733 13013 consensus_queue.cc:260] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.681967 13013 ts_tablet_manager.cc:1434] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260926 19:58:38.682783 13013 tablet_bootstrap.cc:492] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:38.686214 13031 log.cc:1199] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/e20905f83b2e4c7fa409fce38ec5f81e
I20260926 19:58:38.687899 13012 tablet_bootstrap.cc:492] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.688802 13031 ts_tablet_manager.cc:1950] T e20905f83b2e4c7fa409fce38ec5f81e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:38.689549 13119 tablet.cc:1751] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:38.688278 13012 ts_tablet_manager.cc:1403] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.007s
W20260926 19:58:38.702657 12381 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.254:33143, user_credentials={real_user=slave}} blocked reactor thread for 59658.2us
I20260926 19:58:38.706414 13012 raft_consensus.cc:359] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:38.707050 13012 raft_consensus.cc:385] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.708269 13012 raft_consensus.cc:740] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.708861 13012 consensus_queue.cc:260] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:38.710765 13012 ts_tablet_manager.cc:1434] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.020s user 0.004s sys 0.000s
I20260926 19:58:38.711977 13012 tablet_bootstrap.cc:492] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.713696 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet e20905f83b2e4c7fa409fce38ec5f81e (table default.Table_a6745a20e46f46c8bda622354e9d558b [id=c699605165e847f4bca3d95b659eb989]) successfully deleted
I20260926 19:58:38.716418 13142 tablet.cc:1751] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:38.717084 13012 tablet_bootstrap.cc:654] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.732566 13012 tablet_bootstrap.cc:492] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.733002 13012 ts_tablet_manager.cc:1403] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.003s
I20260926 19:58:38.722149 13031 tablet_replica.cc:333] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:38.733810 13031 raft_consensus.cc:2243] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.734583 13031 raft_consensus.cc:2272] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:38.736506 13012 raft_consensus.cc:359] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.737111 13012 raft_consensus.cc:385] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.737380 13012 raft_consensus.cc:740] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.737995 13012 consensus_queue.cc:260] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.739986 13012 ts_tablet_manager.cc:1434] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260926 19:58:38.740806 13012 tablet_bootstrap.cc:492] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.747999 13012 tablet_bootstrap.cc:654] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.744184 13031 ts_tablet_manager.cc:1916] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.758708 13012 tablet_bootstrap.cc:492] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.759094 13012 ts_tablet_manager.cc:1403] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20260926 19:58:38.761252 13012 raft_consensus.cc:359] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.761875 13012 raft_consensus.cc:385] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.762128 13012 raft_consensus.cc:740] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.762701 13012 consensus_queue.cc:260] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.764575 13012 ts_tablet_manager.cc:1434] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 19:58:38.765358 13012 tablet_bootstrap.cc:492] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.768944 13092 tablet_bootstrap.cc:492] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:38.769600 13092 ts_tablet_manager.cc:1403] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.322s user 0.008s sys 0.004s
I20260926 19:58:38.771353 13012 tablet_bootstrap.cc:654] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.772085 13092 raft_consensus.cc:359] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.772738 13092 raft_consensus.cc:385] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.773047 13092 raft_consensus.cc:740] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.782267 13012 tablet_bootstrap.cc:492] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.782675 13012 ts_tablet_manager.cc:1403] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.004s
I20260926 19:58:38.785252 13012 raft_consensus.cc:359] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.785856 13012 raft_consensus.cc:385] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.786096 13012 raft_consensus.cc:740] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.786669 13012 consensus_queue.cc:260] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.789496 13012 ts_tablet_manager.cc:1434] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 19:58:38.789723 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 82f3de5e9b6641508ba56f9b33e930f2
I20260926 19:58:38.790336 13012 tablet_bootstrap.cc:492] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.794703 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 82f3de5e9b6641508ba56f9b33e930f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:38.798586 13012 tablet_bootstrap.cc:654] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.798534 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 82f3de5e9b6641508ba56f9b33e930f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:38.805903 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 82f3de5e9b6641508ba56f9b33e930f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:38.811527 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_dd0aa5f5971b4394a8aa99f77fa7d0eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:38.813282 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd0aa5f5971b4394a8aa99f77fa7d0eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.826107 13031 ts_tablet_manager.cc:1929] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:38.826495 13031 log.cc:1199] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/05a6d163900947689b24489787bc119c
I20260926 19:58:38.827131 13031 ts_tablet_manager.cc:1950] T 05a6d163900947689b24489787bc119c P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:38.829370 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 05a6d163900947689b24489787bc119c (table default.Table_68d42bf187a146958c5daf317378dafd [id=37adf5d8d6c1411b8e6028df63dcd641]) successfully deleted
W20260926 19:58:38.838611 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet e365004bba0d428cbb1900ad558efe5e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e365004bba0d428cbb1900ad558efe5e already in progress: creating tablet
I20260926 19:58:38.840428 13095 raft_consensus.cc:493] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:38.840853 13095 raft_consensus.cc:515] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:38.842672 13095 leader_election.cc:290] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:38.846025 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eca1e5734324f99b9362915bdd00239" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:38.846338 13012 tablet_bootstrap.cc:492] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
W20260926 19:58:38.846365 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet ef062e1775bd4a3c859124eaaba81399 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef062e1775bd4a3c859124eaaba81399 already in progress: creating tablet
I20260926 19:58:38.847153 13012 ts_tablet_manager.cc:1403] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.057s user 0.013s sys 0.027s
W20260926 19:58:38.848172 12696 leader_election.cc:343] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:38.849639 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 4eca1e5734324f99b9362915bdd00239 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4eca1e5734324f99b9362915bdd00239 already in progress: creating tablet
I20260926 19:58:38.850883 13031 tablet_replica.cc:333] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:38.853220 13031 raft_consensus.cc:2243] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.853696 13031 raft_consensus.cc:2272] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:38.861078 13012 raft_consensus.cc:359] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.862182 13012 raft_consensus.cc:385] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.862469 13012 raft_consensus.cc:740] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.863065 13012 consensus_queue.cc:260] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.864905 13012 ts_tablet_manager.cc:1434] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260926 19:58:38.865545 13031 ts_tablet_manager.cc:1916] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.866540 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4eca1e5734324f99b9362915bdd00239" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
W20260926 19:58:38.873803 12695 leader_election.cc:343] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: Tablet not RUNNING: STOPPED
I20260926 19:58:38.874244 12695 leader_election.cc:304] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:38.876454 13128 raft_consensus.cc:493] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:38.876863 13128 raft_consensus.cc:515] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.878713 13128 leader_election.cc:290] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:38.879273 12388 master_service.cc:960] ReplaceTablet: received request to replace tablet 99cf26cf4838454c98aa8337341ffd06 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:38.880357 12392 master_service.cc:960] ReplaceTablet: received request to replace tablet 82f3de5e9b6641508ba56f9b33e930f2 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:38.879922 13095 raft_consensus.cc:2749] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:38.882776 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e365004bba0d428cbb1900ad558efe5e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:38.884250 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e365004bba0d428cbb1900ad558efe5e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:38.885651 12521 leader_election.cc:343] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:38.886693 12519 leader_election.cc:343] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: Tablet not RUNNING: STOPPED
I20260926 19:58:38.887097 12519 leader_election.cc:304] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:38.888165 13128 raft_consensus.cc:2749] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:38.688910 13013 tablet_bootstrap.cc:654] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.882972 13012 tablet_bootstrap.cc:492] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.892370 13092 consensus_queue.cc:260] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.905138 13012 tablet_bootstrap.cc:654] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:38.906078 13060 ts_tablet_manager.cc:1929] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:38.906479 13060 log.cc:1199] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/4eca1e5734324f99b9362915bdd00239
I20260926 19:58:38.907297 13060 ts_tablet_manager.cc:1950] T 4eca1e5734324f99b9362915bdd00239 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:38.910712 13092 ts_tablet_manager.cc:1434] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.141s user 0.003s sys 0.002s
I20260926 19:58:38.928619 13012 tablet_bootstrap.cc:492] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.929010 13012 ts_tablet_manager.cc:1403] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.002s
I20260926 19:58:38.926294 13031 ts_tablet_manager.cc:1929] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260926 19:58:38.930989 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b82d75be43e841e7abcc9e9db800aa2d has no consensus state
I20260926 19:58:38.931228 13031 log.cc:1199] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/82f3de5e9b6641508ba56f9b33e930f2
I20260926 19:58:38.931876 13031 ts_tablet_manager.cc:1950] T 82f3de5e9b6641508ba56f9b33e930f2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:38.932071 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_915e98d154a94aaa86c28d3aa1032d6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.934175 13012 raft_consensus.cc:359] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.934782 13012 raft_consensus.cc:385] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.935020 13012 raft_consensus.cc:740] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.935304 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_033bac717553494ebf0ce0b775144fcf" } new_table_name: "default.Table_1281608b01e84b90b871e71e1adc84b9" modify_external_catalogs: true
I20260926 19:58:38.935644 13012 consensus_queue.cc:260] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.937582 13012 ts_tablet_manager.cc:1434] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260926 19:58:38.938366 13012 tablet_bootstrap.cc:492] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:38.939450 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 82f3de5e9b6641508ba56f9b33e930f2 (table default.Table_3ccdb238d9bb45689fc56322b90e12c3 [id=f53ec6e521114c0c9cc60e7627db131c]) successfully deleted
I20260926 19:58:38.943899 13012 tablet_bootstrap.cc:654] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:38.953909 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_915e98d154a94aaa86c28d3aa1032d6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.958571 13149 raft_consensus.cc:493] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:38.959050 13149 raft_consensus.cc:515] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.962184 13012 tablet_bootstrap.cc:492] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:38.962599 13012 ts_tablet_manager.cc:1403] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.012s
I20260926 19:58:38.966487 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99cf26cf4838454c98aa8337341ffd06" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:38.963538 13013 tablet_bootstrap.cc:492] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:38.967409 13013 ts_tablet_manager.cc:1403] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.285s user 0.002s sys 0.008s
I20260926 19:58:38.967854 13092 tablet_bootstrap.cc:492] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
W20260926 19:58:38.968230 12696 leader_election.cc:343] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:38.968892 13149 leader_election.cc:290] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:38.969817 13013 raft_consensus.cc:359] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.970410 13013 raft_consensus.cc:385] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.970297 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99cf26cf4838454c98aa8337341ffd06" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:38.970662 13013 raft_consensus.cc:740] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:38.971748 12695 leader_election.cc:343] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:38.971827 13012 raft_consensus.cc:359] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.972834 13012 raft_consensus.cc:385] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:38.972754 12695 leader_election.cc:304] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:38.973098 13012 raft_consensus.cc:740] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:38.973296 13032 ts_tablet_manager.cc:1929] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:38.973729 13012 consensus_queue.cc:260] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:38.974732 13095 raft_consensus.cc:2749] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:38.975067 13060 tablet_replica.cc:333] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:38.975733 13012 ts_tablet_manager.cc:1434] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260926 19:58:38.975739 13060 raft_consensus.cc:2243] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.976207 13060 raft_consensus.cc:2272] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:38.981765 13013 consensus_queue.cc:260] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:38.983774 13060 ts_tablet_manager.cc:1916] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:38.985636 13032 log.cc:1199] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/e365004bba0d428cbb1900ad558efe5e
I20260926 19:58:38.989657 13013 ts_tablet_manager.cc:1434] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260926 19:58:38.993253 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f1a85473b0594a75947fbf9a21bd5469"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:38.993954 13032 ts_tablet_manager.cc:1950] T e365004bba0d428cbb1900ad558efe5e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
W20260926 19:58:38.994966 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1a85473b0594a75947fbf9a21bd5469 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:38.996227 13032 tablet_replica.cc:333] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:38.996411 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet e365004bba0d428cbb1900ad558efe5e (table default.Table_9ac55a637c1e41f0bfca50930bd701c0 [id=47b086c8685e4ac19c14784f1691cae1]) successfully deleted
I20260926 19:58:38.996904 13032 raft_consensus.cc:2243] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:38.999118 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 4eca1e5734324f99b9362915bdd00239 (table default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95 [id=4692b08d69b344cdaf9226c62886da70]) successfully deleted
I20260926 19:58:38.999761 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_d2d87877a7c4476aa7e41623c09ed4df" } modify_external_catalogs: true
I20260926 19:58:39.000268 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_d2d87877a7c4476aa7e41623c09ed4df" } modify_external_catalogs: true
I20260926 19:58:39.000495 13013 tablet_bootstrap.cc:492] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:39.012042 12396 catalog_manager.cc:5838] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:39.017002 13013 tablet_bootstrap.cc:654] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.022712 12739 tablet_service.cc:1511] Processing CreateTablet for tablet caf4547f148f49b8943f4e1c7f2132a7 (DEFAULT_TABLE table=default.Table_724fda2a18204a24979779d6e0aafb81 [id=b89cc0604f6f4019802a9ce497b4b954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.023799 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf4547f148f49b8943f4e1c7f2132a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.029688 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e (DEFAULT_TABLE table=default.Table_b1794633093d4c15bfaa7f160828ff99 [id=c6f1da949bf242ae87b8bbdc6c1d334a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.030642 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d25f740c1334c5cb53d7cffa3eb275e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.032648 13013 tablet_bootstrap.cc:492] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:39.033010 13013 ts_tablet_manager.cc:1403] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.000s
I20260926 19:58:39.035727 13013 raft_consensus.cc:359] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.035933 12903 tablet_service.cc:1511] Processing CreateTablet for tablet caf4547f148f49b8943f4e1c7f2132a7 (DEFAULT_TABLE table=default.Table_724fda2a18204a24979779d6e0aafb81 [id=b89cc0604f6f4019802a9ce497b4b954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.036315 13013 raft_consensus.cc:385] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.036537 13013 raft_consensus.cc:740] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.036883 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf4547f148f49b8943f4e1c7f2132a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.037257 13013 consensus_queue.cc:260] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.038097 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e (DEFAULT_TABLE table=default.Table_b1794633093d4c15bfaa7f160828ff99 [id=c6f1da949bf242ae87b8bbdc6c1d334a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:38.997313 13032 raft_consensus.cc:2272] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:39.038949 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d25f740c1334c5cb53d7cffa3eb275e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.044150 13032 ts_tablet_manager.cc:1916] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:39.046159 12567 tablet_service.cc:1511] Processing CreateTablet for tablet caf4547f148f49b8943f4e1c7f2132a7 (DEFAULT_TABLE table=default.Table_724fda2a18204a24979779d6e0aafb81 [id=b89cc0604f6f4019802a9ce497b4b954]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.047089 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf4547f148f49b8943f4e1c7f2132a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.050226 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e (DEFAULT_TABLE table=default.Table_b1794633093d4c15bfaa7f160828ff99 [id=c6f1da949bf242ae87b8bbdc6c1d334a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.051219 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d25f740c1334c5cb53d7cffa3eb275e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.052033 13013 ts_tablet_manager.cc:1434] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260926 19:58:38.993381 13092 tablet_bootstrap.cc:654] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.055950 13013 tablet_bootstrap.cc:492] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:39.066649 13013 tablet_bootstrap.cc:654] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.069690 12393 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_14c589443e094e22bc1c798043fd6d04 [id=01bf90b95d6c402a9dbdf3e5b3f760d6] alter complete (version 3)
W20260926 19:58:39.095582 12387 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b46a5297ef854afd8ef783bf0692135e consensus state has no leader
I20260926 19:58:39.123744 12901 tablet_service.cc:1511] Processing CreateTablet for tablet b82d75be43e841e7abcc9e9db800aa2d (DEFAULT_TABLE table=default.Table_bffd1e391c2f49089149f97d1f0be4e9 [id=22559b05bf1541c09bbc6ae9689bb8fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.124625 13092 tablet_bootstrap.cc:492] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:39.124799 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82d75be43e841e7abcc9e9db800aa2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.127692 12568 tablet_service.cc:1511] Processing CreateTablet for tablet b82d75be43e841e7abcc9e9db800aa2d (DEFAULT_TABLE table=default.Table_bffd1e391c2f49089149f97d1f0be4e9 [id=22559b05bf1541c09bbc6ae9689bb8fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.128687 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82d75be43e841e7abcc9e9db800aa2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.130319 13092 ts_tablet_manager.cc:1403] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.163s user 0.003s sys 0.010s
I20260926 19:58:39.130051 12738 tablet_service.cc:1511] Processing CreateTablet for tablet b82d75be43e841e7abcc9e9db800aa2d (DEFAULT_TABLE table=default.Table_bffd1e391c2f49089149f97d1f0be4e9 [id=22559b05bf1541c09bbc6ae9689bb8fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.131556 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82d75be43e841e7abcc9e9db800aa2d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:39.133993 12380 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet b46a5297ef854afd8ef783bf0692135e (table default.Table_1281608b01e84b90b871e71e1adc84b9 [id=a774283438c64c668a5b2e9e4d62ed99], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b46a5297ef854afd8ef783bf0692135e consensus state has no leader
I20260926 19:58:39.144062 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_bffd1e391c2f49089149f97d1f0be4e9" } new_table_name: "default.Table_fe617b43605c467aa4ded460c57b4624" modify_external_catalogs: true
I20260926 19:58:39.151180 13092 raft_consensus.cc:359] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.155140 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9cf5aa5585a04712ab95e4b0d9428171"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:39.156811 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cf5aa5585a04712ab95e4b0d9428171 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:39.156934 12393 master_service.cc:960] ReplaceTablet: received request to replace tablet 82f3de5e9b6641508ba56f9b33e930f2 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:39.162928 13013 tablet_bootstrap.cc:492] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:39.163314 13013 ts_tablet_manager.cc:1403] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.108s user 0.017s sys 0.018s
I20260926 19:58:39.165484 13013 raft_consensus.cc:359] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.166083 13013 raft_consensus.cc:385] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.166436 13013 raft_consensus.cc:740] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.167145 13013 consensus_queue.cc:260] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.171000 12900 tablet_service.cc:1511] Processing CreateTablet for tablet ec6d07cb4c954f23be80b315a5748adc (DEFAULT_TABLE table=default.Table_56393c731f9e44fe823755a7e91ccc3e [id=137dcda542e04a92938b739a96b65ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.172067 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6d07cb4c954f23be80b315a5748adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.171882 12568 tablet_service.cc:1511] Processing CreateTablet for tablet ec6d07cb4c954f23be80b315a5748adc (DEFAULT_TABLE table=default.Table_56393c731f9e44fe823755a7e91ccc3e [id=137dcda542e04a92938b739a96b65ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.152092 13092 raft_consensus.cc:385] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.172622 13092 raft_consensus.cc:740] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.172792 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6d07cb4c954f23be80b315a5748adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.178191 13092 consensus_queue.cc:260] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.184659 13013 ts_tablet_manager.cc:1434] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.021s user 0.001s sys 0.005s
I20260926 19:58:39.188099 13013 tablet_bootstrap.cc:492] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:39.189040 13092 ts_tablet_manager.cc:1434] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.040s user 0.005s sys 0.001s
I20260926 19:58:39.191342 13092 tablet_bootstrap.cc:492] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:39.208976 12741 tablet_service.cc:1511] Processing CreateTablet for tablet ec6d07cb4c954f23be80b315a5748adc (DEFAULT_TABLE table=default.Table_56393c731f9e44fe823755a7e91ccc3e [id=137dcda542e04a92938b739a96b65ea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.210192 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6d07cb4c954f23be80b315a5748adc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.215425 13060 ts_tablet_manager.cc:1929] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:39.215785 13060 log.cc:1199] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/82f3de5e9b6641508ba56f9b33e930f2
I20260926 19:58:39.216566 13060 ts_tablet_manager.cc:1950] T 82f3de5e9b6641508ba56f9b33e930f2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:39.216629 13092 tablet_bootstrap.cc:654] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.219661 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 381320b4d2844e2289a5ae390e6b0503 (DEFAULT_TABLE table=default.Table_9f3804455bdf4e8f81dcbf449bdb9c03 [id=2206b9a20656421f8bbd96810835c982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.222266 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 28b592db99c1495aafb555ff2a76a154 (DEFAULT_TABLE table=default.Table_f6554f3a9eb54288a5b4b0a316bfbbf6 [id=c2c079dc87ec480a82aa5b57b110da17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.222561 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 381320b4d2844e2289a5ae390e6b0503 (DEFAULT_TABLE table=default.Table_9f3804455bdf4e8f81dcbf449bdb9c03 [id=2206b9a20656421f8bbd96810835c982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.220741 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 381320b4d2844e2289a5ae390e6b0503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.222991 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 28b592db99c1495aafb555ff2a76a154 (DEFAULT_TABLE table=default.Table_f6554f3a9eb54288a5b4b0a316bfbbf6 [id=c2c079dc87ec480a82aa5b57b110da17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.223610 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 381320b4d2844e2289a5ae390e6b0503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.224015 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b592db99c1495aafb555ff2a76a154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.224609 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b592db99c1495aafb555ff2a76a154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.237675 13013 tablet_bootstrap.cc:654] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.241583 13012 tablet_bootstrap.cc:492] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:39.241940 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 1d25f740c1334c5cb53d7cffa3eb275e
I20260926 19:58:39.243613 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:39.244813 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 28b592db99c1495aafb555ff2a76a154 (DEFAULT_TABLE table=default.Table_f6554f3a9eb54288a5b4b0a316bfbbf6 [id=c2c079dc87ec480a82aa5b57b110da17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.246182 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28b592db99c1495aafb555ff2a76a154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.246745 13012 tablet_bootstrap.cc:654] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:39.247591 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 1d25f740c1334c5cb53d7cffa3eb275e because the tablet was not found. No further retry: Not found: Tablet not found: 1d25f740c1334c5cb53d7cffa3eb275e
I20260926 19:58:39.249048 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 99cf26cf4838454c98aa8337341ffd06
I20260926 19:58:39.250542 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:39.250983 12388 catalog_manager.cc:6934] ReplaceTablet: tablet 99cf26cf4838454c98aa8337341ffd06 deleted and replaced by tablet 0ea151df659944f9a094fe9410c3f4bb
I20260926 19:58:39.252049 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 00767b01d2ad4ca5b6cfcc566739f61d
I20260926 19:58:39.252743 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:39.254558 12899 tablet_service.cc:1511] Processing CreateTablet for tablet ffac209a6cd8440289ee7765c3c24b40 (DEFAULT_TABLE table=default.Table_353f84cf17724709af77fc8d316ca0c4 [id=f2217cf0f71142ceb39f447b6e2cc591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.256033 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffac209a6cd8440289ee7765c3c24b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.257817 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 99cf26cf4838454c98aa8337341ffd06 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ea151df659944f9a094fe9410c3f4bb) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:39.260547 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 00767b01d2ad4ca5b6cfcc566739f61d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:33594
W20260926 19:58:39.262928 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 99cf26cf4838454c98aa8337341ffd06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99cf26cf4838454c98aa8337341ffd06 already in progress: creating tablet
W20260926 19:58:39.263621 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 00767b01d2ad4ca5b6cfcc566739f61d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00767b01d2ad4ca5b6cfcc566739f61d already in progress: creating tablet
I20260926 19:58:39.266052 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 99cf26cf4838454c98aa8337341ffd06 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ea151df659944f9a094fe9410c3f4bb) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:39.266135 13032 ts_tablet_manager.cc:1929] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:39.266940 13032 log.cc:1199] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/ef062e1775bd4a3c859124eaaba81399
I20260926 19:58:39.267582 13032 ts_tablet_manager.cc:1950] T ef062e1775bd4a3c859124eaaba81399 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:39.268575 12562 tablet_service.cc:1511] Processing CreateTablet for tablet ffac209a6cd8440289ee7765c3c24b40 (DEFAULT_TABLE table=default.Table_353f84cf17724709af77fc8d316ca0c4 [id=f2217cf0f71142ceb39f447b6e2cc591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.271209 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffac209a6cd8440289ee7765c3c24b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.275282 12902 tablet_service.cc:1511] Processing CreateTablet for tablet da8875d46f7c46819ee67e650b098c27 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.275379 12898 tablet_service.cc:1511] Processing CreateTablet for tablet ba3f3931a71048fd874c1131d44f7b00 (DEFAULT_TABLE table=default.Table_be1467ad5f7e4a92b7f91320650f32bd [id=3dd781a8406045b6b90c7008824c7a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.276521 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8875d46f7c46819ee67e650b098c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.277540 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba3f3931a71048fd874c1131d44f7b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.277683 13032 tablet_replica.cc:333] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:39.278374 13032 raft_consensus.cc:2243] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:39.278807 13032 raft_consensus.cc:2272] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:39.286722 13048 raft_consensus.cc:493] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:39.287156 13048 raft_consensus.cc:515] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.289016 13048 leader_election.cc:290] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.291038 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet 99cf26cf4838454c98aa8337341ffd06 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ea151df659944f9a094fe9410c3f4bb) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:39.291478 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e82262ea1c441808e9765aef911c903" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:39.291900 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet 00767b01d2ad4ca5b6cfcc566739f61d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:39.292230 12761 raft_consensus.cc:2393] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:39.293537 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e82262ea1c441808e9765aef911c903" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:39.294708 12738 tablet_service.cc:1511] Processing CreateTablet for tablet ffac209a6cd8440289ee7765c3c24b40 (DEFAULT_TABLE table=default.Table_353f84cf17724709af77fc8d316ca0c4 [id=f2217cf0f71142ceb39f447b6e2cc591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.295265 12735 tablet_service.cc:1511] Processing CreateTablet for tablet da8875d46f7c46819ee67e650b098c27 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.295876 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffac209a6cd8440289ee7765c3c24b40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.296247 12858 leader_election.cc:304] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:39.297402 13032 ts_tablet_manager.cc:1916] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:39.297300 13048 raft_consensus.cc:3060] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:39.298322 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8875d46f7c46819ee67e650b098c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.303663 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet 4eca1e5734324f99b9362915bdd00239 with delete_type TABLET_DATA_DELETED (replaced by tablet b19f42511ad14d8a88fc0eaefb82e335) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:39.305258 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 82f3de5e9b6641508ba56f9b33e930f2 (table default.Table_3ccdb238d9bb45689fc56322b90e12c3 [id=f53ec6e521114c0c9cc60e7627db131c]) successfully deleted
I20260926 19:58:39.305261 12560 tablet_service.cc:1511] Processing CreateTablet for tablet da8875d46f7c46819ee67e650b098c27 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.305292 13048 raft_consensus.cc:2749] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:39.305981 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 1d25f740c1334c5cb53d7cffa3eb275e because the tablet was not found. No further retry: Not found: Tablet not found: 1d25f740c1334c5cb53d7cffa3eb275e
I20260926 19:58:39.306391 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8875d46f7c46819ee67e650b098c27. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:39.306878 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 99cf26cf4838454c98aa8337341ffd06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99cf26cf4838454c98aa8337341ffd06 already in progress: creating tablet
W20260926 19:58:39.307554 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 4eca1e5734324f99b9362915bdd00239 because the tablet was not found. No further retry: Not found: Tablet not found: 4eca1e5734324f99b9362915bdd00239
I20260926 19:58:39.309980 12561 tablet_service.cc:1511] Processing CreateTablet for tablet ba3f3931a71048fd874c1131d44f7b00 (DEFAULT_TABLE table=default.Table_be1467ad5f7e4a92b7f91320650f32bd [id=3dd781a8406045b6b90c7008824c7a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.311446 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba3f3931a71048fd874c1131d44f7b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.324913 13013 tablet_bootstrap.cc:492] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:39.325349 13013 ts_tablet_manager.cc:1403] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.137s user 0.011s sys 0.002s
I20260926 19:58:39.321095 12737 tablet_service.cc:1511] Processing CreateTablet for tablet ba3f3931a71048fd874c1131d44f7b00 (DEFAULT_TABLE table=default.Table_be1467ad5f7e4a92b7f91320650f32bd [id=3dd781a8406045b6b90c7008824c7a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.326474 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba3f3931a71048fd874c1131d44f7b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.329057 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 381320b4d2844e2289a5ae390e6b0503 (DEFAULT_TABLE table=default.Table_9f3804455bdf4e8f81dcbf449bdb9c03 [id=2206b9a20656421f8bbd96810835c982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:39.330108 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet ef062e1775bd4a3c859124eaaba81399 (table default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]) successfully deleted
I20260926 19:58:39.335007 13149 raft_consensus.cc:493] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:39.335457 13149 raft_consensus.cc:515] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.339450 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1822d4c697d4c4985902355fc360dd1" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:39.341603 12696 leader_election.cc:343] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.342319 13149 leader_election.cc:290] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.346817 13149 raft_consensus.cc:493] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:39.347229 13149 raft_consensus.cc:515] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.350732 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c683dc1272f4290a56f7fc7010ba038" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:39.352185 12696 leader_election.cc:343] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.352196 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_2c324c2f21db4f39a507bf9b47ff489d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:39.352749 13149 leader_election.cc:290] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.354202 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f6554f3a9eb54288a5b4b0a316bfbbf6" } new_table_name: "default.Table_5e21794a58a648c8a5806d4c535a8f37" modify_external_catalogs: true
I20260926 19:58:39.357693 13012 tablet_bootstrap.cc:492] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:39.358215 13012 ts_tablet_manager.cc:1403] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.117s user 0.014s sys 0.018s
I20260926 19:58:39.361882 13012 raft_consensus.cc:359] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.362176 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_0c83b5f2aca049df9f95b55670a04502"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:39.362644 13012 raft_consensus.cc:385] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.362936 13012 raft_consensus.cc:740] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
W20260926 19:58:39.363914 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c83b5f2aca049df9f95b55670a04502 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:39.363824 13012 consensus_queue.cc:260] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.369004 13012 ts_tablet_manager.cc:1434] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260926 19:58:39.370352 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c324c2f21db4f39a507bf9b47ff489d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:39.371562 13012 tablet_bootstrap.cc:492] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:39.379307 13012 tablet_bootstrap.cc:654] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.379279 12395 catalog_manager.cc:5838] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:39.393064 13092 tablet_bootstrap.cc:492] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:39.393934 13092 ts_tablet_manager.cc:1403] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.203s user 0.011s sys 0.001s
I20260926 19:58:39.396550 13092 raft_consensus.cc:359] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.397964 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 6f1dafade40a46649ac2780b8ad3b9ff (table default.Table_b3e35b31063d4ba7a63fc5af3f5dd752 [id=e8e5454a4f9243988f48a9c7d26a7736]). Expected version 1 got 0
I20260926 19:58:39.411413 12565 tablet_service.cc:1558] Processing DeleteTablet for tablet 00767b01d2ad4ca5b6cfcc566739f61d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:39.412866 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_0501c3819754464dbfeecf1cc22595ea [id=b1955ec80c98477c95b3b06cb73f3206]). Expected version 1 got 0
I20260926 19:58:39.414113 13060 tablet_replica.cc:333] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:39.415079 13060 raft_consensus.cc:2243] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:39.415648 13060 raft_consensus.cc:2272] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:39.418381 13060 ts_tablet_manager.cc:1916] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:39.424544 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 381320b4d2844e2289a5ae390e6b0503. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:39.425724 13128 consensus_queue.cc:1048] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:39.428573 13013 raft_consensus.cc:359] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.429201 13013 raft_consensus.cc:385] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.429454 13013 raft_consensus.cc:740] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.429554 12393 master_service.cc:960] ReplaceTablet: received request to replace tablet 00767b01d2ad4ca5b6cfcc566739f61d from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:39.430073 13013 consensus_queue.cc:260] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.425175 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet d38d785b63734ddd9f2e4f47ef55c7c9 (table default.Table_ed4e65d4240d48f88a62a3b5e15174bd [id=244a65d5edb64b3b8b8ca011dfbc9e24]). Expected version 1 got 0
I20260926 19:58:39.436640 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1822d4c697d4c4985902355fc360dd1" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:39.437250 12594 raft_consensus.cc:2468] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:39.441761 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c683dc1272f4290a56f7fc7010ba038" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:39.442991 12695 leader_election.cc:304] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:39.444023 13149 raft_consensus.cc:2804] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:39.444348 13149 raft_consensus.cc:493] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:39.444664 13149 raft_consensus.cc:3060] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:39.447631 13013 ts_tablet_manager.cc:1434] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.122s user 0.006s sys 0.000s
I20260926 19:58:39.447721 13095 raft_consensus.cc:493] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:39.448150 13095 raft_consensus.cc:515] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.450012 13095 leader_election.cc:290] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.452126 13060 ts_tablet_manager.cc:1929] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:39.451704 13149 raft_consensus.cc:515] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.452108 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46a5297ef854afd8ef783bf0692135e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:39.452507 13060 log.cc:1199] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/00767b01d2ad4ca5b6cfcc566739f61d
I20260926 19:58:39.453168 13060 ts_tablet_manager.cc:1950] T 00767b01d2ad4ca5b6cfcc566739f61d P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
W20260926 19:58:39.455595 12696 leader_election.cc:343] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.456398 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1822d4c697d4c4985902355fc360dd1" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:39.452270 13013 tablet_bootstrap.cc:492] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
W20260926 19:58:39.463225 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b82d75be43e841e7abcc9e9db800aa2d consensus state has no leader
W20260926 19:58:39.466756 12381 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.254:33143, user_credentials={real_user=slave}} blocked reactor thread for 57341.7us
W20260926 19:58:39.476866 12695 leader_election.cc:343] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.477334 12695 leader_election.cc:304] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:39.480755 12387 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_36db24f73aca410799bcb881e2bd1bf9" } modify_external_catalogs: true
I20260926 19:58:39.481484 12387 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_36db24f73aca410799bcb881e2bd1bf9" } modify_external_catalogs: true
W20260926 19:58:39.484795 12696 leader_election.cc:343] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.486402 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b3e35b31063d4ba7a63fc5af3f5dd752" } modify_external_catalogs: true
I20260926 19:58:39.487105 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b3e35b31063d4ba7a63fc5af3f5dd752" } modify_external_catalogs: true
I20260926 19:58:39.487598 13013 tablet_bootstrap.cc:654] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.488000 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 00767b01d2ad4ca5b6cfcc566739f61d (table default.Table_d2d87877a7c4476aa7e41623c09ed4df [id=d4475cbaf3a74d19818f03cdb5ca6537]) successfully deleted
I20260926 19:58:39.489454 13149 leader_election.cc:290] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.496986 12389 master_service.cc:960] ReplaceTablet: received request to replace tablet ef062e1775bd4a3c859124eaaba81399 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:39.502065 13032 ts_tablet_manager.cc:1929] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:39.505688 13032 log.cc:1199] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/4eca1e5734324f99b9362915bdd00239
I20260926 19:58:39.506450 13032 ts_tablet_manager.cc:1950] T 4eca1e5734324f99b9362915bdd00239 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:39.508863 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 4eca1e5734324f99b9362915bdd00239 (table default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95 [id=4692b08d69b344cdaf9226c62886da70]) successfully deleted
I20260926 19:58:39.489974 13149 raft_consensus.cc:2749] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:39.515568 13032 tablet_replica.cc:333] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:39.516374 13032 raft_consensus.cc:2243] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:39.520538 13032 raft_consensus.cc:2272] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:39.521899 13149 raft_consensus.cc:493] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:39.523205 13095 raft_consensus.cc:493] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:39.523622 13095 raft_consensus.cc:515] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.524729 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46a5297ef854afd8ef783bf0692135e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:39.525301 12594 raft_consensus.cc:2468] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:39.527019 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1822d4c697d4c4985902355fc360dd1" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:39.527024 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38d785b63734ddd9f2e4f47ef55c7c9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:39.527508 12594 raft_consensus.cc:3060] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:39.528514 12696 leader_election.cc:343] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.529176 13095 leader_election.cc:290] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.522382 13149 raft_consensus.cc:515] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.532184 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00767b01d2ad4ca5b6cfcc566739f61d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:39.533558 12696 leader_election.cc:343] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.535380 13149 leader_election.cc:290] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.542769 12695 leader_election.cc:304] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:39.536356 13032 ts_tablet_manager.cc:1916] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:39.548480 13149 raft_consensus.cc:2804] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:39.548904 13149 raft_consensus.cc:493] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:39.549312 13149 raft_consensus.cc:3060] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:39.555961 13149 raft_consensus.cc:515] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.560360 13165 consensus_queue.cc:1048] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:39.573674 13165 consensus_queue.cc:1048] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260926 19:58:39.578828 12521 consensus_peers.cc:597] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:39.584801 12594 raft_consensus.cc:2468] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
W20260926 19:58:39.612792 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28b592db99c1495aafb555ff2a76a154 consensus state has no leader
I20260926 19:58:39.619740 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet d38d785b63734ddd9f2e4f47ef55c7c9 (table default.Table_ed4e65d4240d48f88a62a3b5e15174bd [id=244a65d5edb64b3b8b8ca011dfbc9e24]). Expected version 1 got 0
I20260926 19:58:39.620510 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet b46a5297ef854afd8ef783bf0692135e (table default.Table_1281608b01e84b90b871e71e1adc84b9 [id=a774283438c64c668a5b2e9e4d62ed99]). Expected version 1 got 0
I20260926 19:58:39.623309 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38d785b63734ddd9f2e4f47ef55c7c9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:39.623930 12595 raft_consensus.cc:2468] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:39.626473 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet e365004bba0d428cbb1900ad558efe5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:39.627333 13031 tablet_replica.cc:333] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:39.628201 13031 raft_consensus.cc:2243] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:39.628746 13031 raft_consensus.cc:2272] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:39.631480 13031 ts_tablet_manager.cc:1916] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:39.397269 13092 raft_consensus.cc:385] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.648671 13092 raft_consensus.cc:740] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.647666 13031 ts_tablet_manager.cc:1929] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:39.655833 13031 log.cc:1199] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/e365004bba0d428cbb1900ad558efe5e
I20260926 19:58:39.656989 13031 ts_tablet_manager.cc:1950] T e365004bba0d428cbb1900ad558efe5e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:39.657948 13092 consensus_queue.cc:260] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.660552 13092 ts_tablet_manager.cc:1434] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.266s user 0.005s sys 0.000s
I20260926 19:58:39.671716 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet e365004bba0d428cbb1900ad558efe5e (table default.Table_9ac55a637c1e41f0bfca50930bd701c0 [id=47b086c8685e4ac19c14784f1691cae1]) successfully deleted
I20260926 19:58:39.684612 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_3c2a281afe824c4b9cc42a61d8b4ed1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:39.686305 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c2a281afe824c4b9cc42a61d8b4ed1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:39.689774 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_ed4e65d4240d48f88a62a3b5e15174bd" } new_table_name: "default.Table_199fcdc45b23492aacc9631f50874ddb" modify_external_catalogs: true
I20260926 19:58:39.693434 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f1e412eb951f446990776744666978fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:39.695087 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1e412eb951f446990776744666978fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:39.629200 13095 raft_consensus.cc:493] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:39.722779 13095 raft_consensus.cc:515] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.724555 13095 leader_election.cc:290] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.661427 13092 tablet_bootstrap.cc:492] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:39.630635 12695 leader_election.cc:304] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
W20260926 19:58:39.725914 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 95331.7us
I20260926 19:58:39.727324 12695 leader_election.cc:304] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:39.727746 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9acbd4e5aa23433d860d399d78485d0d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:39.728341 12695 leader_election.cc:343] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 00767b01d2ad4ca5b6cfcc566739f61d
I20260926 19:58:39.728742 12695 leader_election.cc:304] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:39.729846 13095 raft_consensus.cc:2804] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
W20260926 19:58:39.731223 12696 leader_election.cc:343] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.712646 13012 tablet_bootstrap.cc:492] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:39.737836 13012 ts_tablet_manager.cc:1403] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.366s user 0.079s sys 0.182s
I20260926 19:58:39.739961 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46a5297ef854afd8ef783bf0692135e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:39.742878 12696 leader_election.cc:343] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.744392 13149 leader_election.cc:290] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.745031 13149 raft_consensus.cc:2804] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:39.745369 13149 raft_consensus.cc:493] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:39.745762 13149 raft_consensus.cc:3060] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:39.755841 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9acbd4e5aa23433d860d399d78485d0d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:39.755971 13012 raft_consensus.cc:359] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.756623 13012 raft_consensus.cc:385] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.756876 13012 raft_consensus.cc:740] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.765583 13013 tablet_bootstrap.cc:492] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:39.765969 13013 ts_tablet_manager.cc:1403] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.314s user 0.058s sys 0.128s
I20260926 19:58:39.768250 13013 raft_consensus.cc:359] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.768906 13013 raft_consensus.cc:385] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.769202 13013 raft_consensus.cc:740] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.774250 13176 raft_consensus.cc:2749] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:39.780674 13092 tablet_bootstrap.cc:654] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.757491 13012 consensus_queue.cc:260] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.787210 13012 ts_tablet_manager.cc:1434] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260926 19:58:39.788218 13012 tablet_bootstrap.cc:492] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:39.798467 13013 consensus_queue.cc:260] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.799306 12393 master_service.cc:960] ReplaceTablet: received request to replace tablet 28b592db99c1495aafb555ff2a76a154 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:39.800540 13013 ts_tablet_manager.cc:1434] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.034s user 0.004s sys 0.000s
I20260926 19:58:39.801308 13013 tablet_bootstrap.cc:492] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:39.805546 13032 ts_tablet_manager.cc:1929] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:39.805927 13032 log.cc:1199] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/82f3de5e9b6641508ba56f9b33e930f2
I20260926 19:58:39.806761 13032 ts_tablet_manager.cc:1950] T 82f3de5e9b6641508ba56f9b33e930f2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:39.809134 13032 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 47) took 1016 ms. Trace:
I20260926 19:58:39.814273 13149 raft_consensus.cc:515] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.817907 13149 leader_election.cc:290] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.817788 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38d785b63734ddd9f2e4f47ef55c7c9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:39.819247 12696 leader_election.cc:343] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.823900 13095 raft_consensus.cc:697] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:39.824950 13095 consensus_queue.cc:237] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.827158 13013 tablet_bootstrap.cc:654] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.837867 13092 tablet_bootstrap.cc:492] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:39.838269 13092 ts_tablet_manager.cc:1403] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.177s user 0.009s sys 0.003s
I20260926 19:58:39.841133 13092 raft_consensus.cc:359] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.841732 13092 raft_consensus.cc:385] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.841979 13092 raft_consensus.cc:740] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.844414 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_915e98d154a94aaa86c28d3aa1032d6c" } new_table_name: "default.Table_b68b74a1774041749a10f4006f15ece1" modify_external_catalogs: true
I20260926 19:58:39.848488 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_2b4f759d68744ea8b7d986616f8618d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:39.850121 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b4f759d68744ea8b7d986616f8618d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:39.826066 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 97753.3us
I20260926 19:58:39.831090 13032 rpcz_store.cc:276] 0926 19:58:38.792293 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:38.805692 (+ 13399us) service_pool.cc:224] Handling call
0926 19:58:39.515418 (+709726us) ts_tablet_manager.cc:1151] Deleting tablet 82f3de5e9b6641508ba56f9b33e930f2
0926 19:58:39.515458 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:39.707121 (+191663us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:39.765387 (+ 58266us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:39.809072 (+ 43685us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":708928}
I20260926 19:58:39.849287 13012 tablet_bootstrap.cc:654] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.856256 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 82f3de5e9b6641508ba56f9b33e930f2 (table default.Table_3ccdb238d9bb45689fc56322b90e12c3 [id=f53ec6e521114c0c9cc60e7627db131c]) successfully deleted
W20260926 19:58:39.858021 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 1d25f740c1334c5cb53d7cffa3eb275e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1d25f740c1334c5cb53d7cffa3eb275e already in progress: creating tablet
I20260926 19:58:39.859009 13032 tablet_replica.cc:333] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:39.859810 13032 raft_consensus.cc:2243] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:39.860307 13032 raft_consensus.cc:2272] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:39.863132 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d38d785b63734ddd9f2e4f47ef55c7c9 consensus state has no leader
I20260926 19:58:39.868391 12392 master_service.cc:960] ReplaceTablet: received request to replace tablet ffac209a6cd8440289ee7765c3c24b40 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:39.869910 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_2dd176d225964a3c827572433ed19d99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:39.871282 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2dd176d225964a3c827572433ed19d99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:39.875593 13013 tablet_bootstrap.cc:492] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:39.875730 13032 ts_tablet_manager.cc:1916] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:39.875967 13013 ts_tablet_manager.cc:1403] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.075s user 0.008s sys 0.005s
I20260926 19:58:39.879110 13012 tablet_bootstrap.cc:492] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:39.879503 13012 ts_tablet_manager.cc:1403] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.002s
I20260926 19:58:39.883381 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46a5297ef854afd8ef783bf0692135e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:39.883908 12595 raft_consensus.cc:3060] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:39.884014 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 732a34f4894940b3889bab5f0c0891e0 has no consensus state
I20260926 19:58:39.886996 13013 raft_consensus.cc:359] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.887635 13013 raft_consensus.cc:385] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.887878 13013 raft_consensus.cc:740] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.890700 13013 consensus_queue.cc:260] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.890859 13012 raft_consensus.cc:359] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.891549 13012 raft_consensus.cc:385] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.891784 13012 raft_consensus.cc:740] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.893340 13012 consensus_queue.cc:260] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.893630 12595 raft_consensus.cc:2468] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:39.892522 13092 consensus_queue.cc:260] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.902444 13013 ts_tablet_manager.cc:1434] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.026s user 0.005s sys 0.001s
I20260926 19:58:39.906540 13092 ts_tablet_manager.cc:1434] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.068s user 0.006s sys 0.000s
W20260926 19:58:39.909909 12695 leader_election.cc:343] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.911198 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38d785b63734ddd9f2e4f47ef55c7c9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:39.911831 12595 raft_consensus.cc:3060] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:39.914479 12695 leader_election.cc:304] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:39.907351 13092 tablet_bootstrap.cc:492] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:39.927145 13013 tablet_bootstrap.cc:492] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:39.928222 12695 leader_election.cc:304] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:39.928862 13176 raft_consensus.cc:2749] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:39.915287 13012 ts_tablet_manager.cc:1434] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.036s user 0.007s sys 0.000s
I20260926 19:58:39.930291 13012 tablet_bootstrap.cc:492] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:39.933071 13013 tablet_bootstrap.cc:654] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.941875 12595 raft_consensus.cc:2468] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:39.952934 13092 tablet_bootstrap.cc:654] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.955420 13176 raft_consensus.cc:2804] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:39.956005 13176 raft_consensus.cc:697] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:39.956990 13012 tablet_bootstrap.cc:654] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.957062 13176 consensus_queue.cc:237] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.964675 13180 raft_consensus.cc:493] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:39.965164 13180 raft_consensus.cc:515] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.967163 13013 tablet_bootstrap.cc:492] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:39.967582 13013 ts_tablet_manager.cc:1403] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.000s
I20260926 19:58:39.968355 13180 leader_election.cc:290] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:39.969008 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d75be43e841e7abcc9e9db800aa2d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:39.969930 13013 raft_consensus.cc:359] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.970517 13013 raft_consensus.cc:385] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.970767 13013 raft_consensus.cc:740] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.970511 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_5b2c3bb01e984005a49cb679fa197189"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:39.971027 12696 leader_election.cc:343] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:39.971580 13092 tablet_bootstrap.cc:492] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:39.971465 13013 consensus_queue.cc:260] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:39.971966 13092 ts_tablet_manager.cc:1403] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.000s
W20260926 19:58:39.971936 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b2c3bb01e984005a49cb679fa197189 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:39.973207 13013 ts_tablet_manager.cc:1434] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 19:58:39.973105 12695 leader_election.cc:304] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:39.982251 13092 raft_consensus.cc:359] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.982824 13092 raft_consensus.cc:385] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:39.983057 13092 raft_consensus.cc:740] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:39.986846 13013 tablet_bootstrap.cc:492] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:39.992159 13013 tablet_bootstrap.cc:654] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:39.993657 13092 consensus_queue.cc:260] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:39.995069 13180 raft_consensus.cc:2804] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:39.999691 13092 ts_tablet_manager.cc:1434] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20260926 19:58:40.000499 13092 tablet_bootstrap.cc:492] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:39.996083 13180 raft_consensus.cc:697] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:40.002673 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d75be43e841e7abcc9e9db800aa2d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:40.003131 13180 consensus_queue.cc:237] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.006879 13012 tablet_bootstrap.cc:492] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
W20260926 19:58:40.007140 12695 leader_election.cc:343] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.007288 13092 tablet_bootstrap.cc:654] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.007295 13012 ts_tablet_manager.cc:1403] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.077s user 0.005s sys 0.005s
I20260926 19:58:40.007541 12695 leader_election.cc:304] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:40.009477 13095 raft_consensus.cc:2749] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:40.010648 13012 raft_consensus.cc:359] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.011519 13012 raft_consensus.cc:385] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.011857 13012 raft_consensus.cc:740] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.012670 13012 consensus_queue.cc:260] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.028983 13013 tablet_bootstrap.cc:492] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:40.029487 13013 ts_tablet_manager.cc:1403] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.043s user 0.009s sys 0.004s
I20260926 19:58:40.031879 13013 raft_consensus.cc:359] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.032665 13013 raft_consensus.cc:385] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.032784 13012 ts_tablet_manager.cc:1434] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.025s user 0.003s sys 0.003s
I20260926 19:58:40.032958 13013 raft_consensus.cc:740] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.033685 13012 tablet_bootstrap.cc:492] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:40.033744 13013 consensus_queue.cc:260] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.041384 13013 ts_tablet_manager.cc:1434] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.012s user 0.002s sys 0.002s
I20260926 19:58:40.042717 13013 tablet_bootstrap.cc:492] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:40.049980 13048 raft_consensus.cc:493] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.050530 13048 raft_consensus.cc:515] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:40.052572 13048 leader_election.cc:290] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:40.054872 13032 ts_tablet_manager.cc:1929] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:40.054890 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1822d4c697d4c4985902355fc360dd1" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:40.055228 13032 log.cc:1199] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/99cf26cf4838454c98aa8337341ffd06
I20260926 19:58:40.055652 12595 raft_consensus.cc:2393] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:40.055385 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1822d4c697d4c4985902355fc360dd1" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:40.055840 13032 ts_tablet_manager.cc:1950] T 99cf26cf4838454c98aa8337341ffd06 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:40.058048 12857 leader_election.cc:304] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:40.059563 13048 raft_consensus.cc:3060] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.068583 13012 tablet_bootstrap.cc:654] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.070160 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_0501c3819754464dbfeecf1cc22595ea" } new_table_name: "default.Table_1305c705a9be4c02863b9f3d5d014850" modify_external_catalogs: true
I20260926 19:58:40.071472 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_6cabacdbfcc94b7e876d1a62734db5a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:40.073011 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cabacdbfcc94b7e876d1a62734db5a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:40.077211 13013 tablet_bootstrap.cc:654] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.085644 13048 raft_consensus.cc:2749] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:40.092739 13013 tablet_bootstrap.cc:492] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:40.093120 13013 ts_tablet_manager.cc:1403] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.051s user 0.011s sys 0.005s
I20260926 19:58:40.092898 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3c2a281afe824c4b9cc42a61d8b4ed1c" } new_table_name: "default.Table_124ad4c229ac49ccbc0243da96985a9e" modify_external_catalogs: true
I20260926 19:58:40.095630 13013 raft_consensus.cc:359] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.096236 13013 raft_consensus.cc:385] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.096482 13013 raft_consensus.cc:740] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.097045 13013 consensus_queue.cc:260] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.101050 13013 ts_tablet_manager.cc:1434] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
W20260926 19:58:40.106474 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c1822d4c697d4c4985902355fc360dd1 consensus state has no leader
I20260926 19:58:40.106127 13013 tablet_bootstrap.cc:492] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:40.123353 13128 raft_consensus.cc:493] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.123778 13128 raft_consensus.cc:515] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.132022 13128 leader_election.cc:290] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:40.134473 12519 leader_election.cc:343] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 99cf26cf4838454c98aa8337341ffd06
I20260926 19:58:40.135933 13013 tablet_bootstrap.cc:654] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.139652 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99cf26cf4838454c98aa8337341ffd06" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:40.141338 12521 leader_election.cc:343] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.141984 12521 leader_election.cc:304] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.143312 13128 raft_consensus.cc:2749] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:40.149314 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 6f1dafade40a46649ac2780b8ad3b9ff
I20260926 19:58:40.152534 12735 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f1dafade40a46649ac2780b8ad3b9ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:40.154289 12382 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_1305c705a9be4c02863b9f3d5d014850 [id=b1955ec80c98477c95b3b06cb73f3206], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet c1822d4c697d4c4985902355fc360dd1 consensus state has no leader
I20260926 19:58:40.155295 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f1dafade40a46649ac2780b8ad3b9ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:40.155993 13031 tablet_replica.cc:333] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:40.156622 13031 raft_consensus.cc:2243] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:40.157048 13031 raft_consensus.cc:2272] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.160679 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 99cf26cf4838454c98aa8337341ffd06 (table default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]) successfully deleted
W20260926 19:58:40.162894 12395 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 27) took 831 ms (client timeout 1000 ms). Trace:
I20260926 19:58:40.163985 13031 ts_tablet_manager.cc:1916] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:40.155345 13128 raft_consensus.cc:493] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.180193 13032 tablet_replica.cc:333] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:40.181100 13032 raft_consensus.cc:2243] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:40.181658 13032 raft_consensus.cc:2272] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.159374 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f1dafade40a46649ac2780b8ad3b9ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.162231 13013 tablet_bootstrap.cc:492] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:40.182416 13013 ts_tablet_manager.cc:1403] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.076s user 0.008s sys 0.004s
I20260926 19:58:40.183391 13032 ts_tablet_manager.cc:1916] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:40.184355 13203 tablet_replica.cc:333] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:40.185154 13203 raft_consensus.cc:2243] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:40.185011 13013 raft_consensus.cc:359] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.185706 13013 raft_consensus.cc:385] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.185242 13128 raft_consensus.cc:515] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.186080 13203 raft_consensus.cc:2272] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.185958 13013 raft_consensus.cc:740] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.190618 13092 tablet_bootstrap.cc:492] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:40.191004 13092 ts_tablet_manager.cc:1403] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.191s user 0.020s sys 0.068s
I20260926 19:58:40.194269 13128 leader_election.cc:290] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:40.194591 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 00767b01d2ad4ca5b6cfcc566739f61d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.170323 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_65a24dd219044e0f80e841c1254730d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:40.195108 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet e365004bba0d428cbb1900ad558efe5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.193148 13092 raft_consensus.cc:359] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:40.195835 13092 raft_consensus.cc:385] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:58:40.196210 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65a24dd219044e0f80e841c1254730d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:40.196072 13092 raft_consensus.cc:740] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.197330 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c683dc1272f4290a56f7fc7010ba038" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:40.187645 13013 consensus_queue.cc:260] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.198267 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c683dc1272f4290a56f7fc7010ba038" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:40.198807 12521 leader_election.cc:343] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.198863 12760 raft_consensus.cc:2468] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:40.200232 12519 leader_election.cc:304] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.202198 13173 raft_consensus.cc:2804] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:40.202553 13173 raft_consensus.cc:493] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:40.202857 13173 raft_consensus.cc:3060] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.171639 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a37fe86f467e42e59cecf9922dd1f820"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:40.191692 12382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260926 19:58:40.209867 13092 consensus_queue.cc:260] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:40.211155 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a37fe86f467e42e59cecf9922dd1f820 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:40.212069 13203 ts_tablet_manager.cc:1916] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:40.217080 13092 ts_tablet_manager.cc:1434] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260926 19:58:40.218008 13092 tablet_bootstrap.cc:492] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:40.200203 13013 ts_tablet_manager.cc:1434] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260926 19:58:40.222326 12900 tablet_service.cc:1511] Processing CreateTablet for tablet ed45552637024bec99a76da079f352ec (DEFAULT_TABLE table=default.Table_dd0aa5f5971b4394a8aa99f77fa7d0eb [id=667dce3292c94f53b899dcba291d56a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.222592 13013 tablet_bootstrap.cc:492] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:40.223351 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed45552637024bec99a76da079f352ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.227115 13173 raft_consensus.cc:515] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.229240 13128 raft_consensus.cc:493] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.229669 13128 raft_consensus.cc:515] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.233606 13128 leader_election.cc:290] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:40.194891 12395 rpcz_store.cc:269] 0926 19:58:39.331637 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:39.331745 (+ 108us) service_pool.cc:224] Handling call
0926 19:58:39.358396 (+ 26651us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:40.162834 (+804438us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:39.456992 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:39.457395 (+ 403us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:39.457418 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:58:39.457439 (+ 21us) tablet.cc:662] Decoding operations
0926 19:58:39.457721 (+ 282us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:39.457865 (+ 144us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:39.457889 (+ 24us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:39.458034 (+ 145us) tablet.cc:701] Row locks acquired
0926 19:58:39.458048 (+ 14us) write_op.cc:260] PREPARE: finished
0926 19:58:39.458155 (+ 107us) write_op.cc:270] Start()
0926 19:58:39.458336 (+ 181us) write_op.cc:276] Timestamp: P: 1790452719458104 usec, L: 0
0926 19:58:39.458351 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:39.463770 (+ 5419us) log.cc:844] Serialized 535 byte log entry
0926 19:58:39.464325 (+ 555us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36ac5a5e8 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11 00007fb36b0edd3d 00000000005bd4e4 00000000005bd319
0926 19:58:39.598380 (+134055us) op_driver.cc:464] REPLICATION: finished
0926 19:58:39.606332 (+ 7952us) write_op.cc:301] APPLY: starting
0926 19:58:39.606381 (+ 49us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:39.606469 (+ 88us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:39.606487 (+ 18us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:39.606623 (+ 136us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:39.606787 (+ 164us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:39.606825 (+ 38us) write_op.cc:312] APPLY: finished
0926 19:58:39.606991 (+ 166us) log.cc:844] Serialized 22 byte log entry
0926 19:58:39.607141 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:39.607214 (+ 73us) write_op.cc:454] Released schema lock
0926 19:58:39.607252 (+ 38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":5735,"reported_tablets":6,"spinlock_wait_cycles":56902912,"child_traces":[["op",{"apply.queue_time_us":7818,"apply.run_cpu_time_us":2839,"apply.run_wall_time_us":6325,"num_ops":1,"prepare.queue_time_us":23844,"prepare.run_cpu_time_us":2496,"prepare.run_wall_time_us":7515,"raft.queue_time_us":180,"raft.run_cpu_time_us":18,"raft.run_wall_time_us":18,"replication_time_us":139958,"spinlock_wait_cycles":11426688,"thread_start_us":65835,"threads_started":1}]]}
I20260926 19:58:40.236394 13173 leader_election.cc:290] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:40.240823 12388 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_1305c705a9be4c02863b9f3d5d014850 [id=b1955ec80c98477c95b3b06cb73f3206]). Expected version 2 got 0
I20260926 19:58:40.244753 13092 tablet_bootstrap.cc:654] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:40.246863 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 27ccbbe1a5784e97827a31e9febd8c20 has no consensus state
W20260926 19:58:40.249208 12394 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 28) took 904 ms (client timeout 1000 ms). Trace:
I20260926 19:58:40.250736 13013 tablet_bootstrap.cc:654] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:40.253167 12394 rpcz_store.cc:269] 0926 19:58:39.344642 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:39.344757 (+ 115us) service_pool.cc:224] Handling call
0926 19:58:39.365920 (+ 21163us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:40.249149 (+883229us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":9955584}
I20260926 19:58:40.252753 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 00767b01d2ad4ca5b6cfcc566739f61d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:40.264351 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c683dc1272f4290a56f7fc7010ba038" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:40.264940 13013 tablet_bootstrap.cc:492] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:40.265300 13013 ts_tablet_manager.cc:1403] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.002s
I20260926 19:58:40.249352 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a9b6637851b849c38be655d4ad6e9a5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:40.245299 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 0ea151df659944f9a094fe9410c3f4bb
W20260926 19:58:40.266798 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9b6637851b849c38be655d4ad6e9a5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:40.271350 13013 raft_consensus.cc:359] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.271677 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ea151df659944f9a094fe9410c3f4bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.271941 13013 raft_consensus.cc:385] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.272179 13013 raft_consensus.cc:740] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.272588 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ea151df659944f9a094fe9410c3f4bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:40.272723 13013 consensus_queue.cc:260] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.277017 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 4eca1e5734324f99b9362915bdd00239 with delete_type TABLET_DATA_DELETED (replaced by tablet b19f42511ad14d8a88fc0eaefb82e335) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:40.277030 13092 tablet_bootstrap.cc:492] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:40.279047 13092 ts_tablet_manager.cc:1403] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.061s user 0.006s sys 0.005s
I20260926 19:58:40.279572 12389 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet b46a5297ef854afd8ef783bf0692135e (table default.Table_1281608b01e84b90b871e71e1adc84b9 [id=a774283438c64c668a5b2e9e4d62ed99]). Expected version 1 got 0
I20260926 19:58:40.280716 12389 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet b82d75be43e841e7abcc9e9db800aa2d (table default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa]). Expected version 2 got 0
I20260926 19:58:40.287849 12562 tablet_service.cc:1511] Processing CreateTablet for tablet ed45552637024bec99a76da079f352ec (DEFAULT_TABLE table=default.Table_dd0aa5f5971b4394a8aa99f77fa7d0eb [id=667dce3292c94f53b899dcba291d56a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.288939 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed45552637024bec99a76da079f352ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.288703 13092 raft_consensus.cc:359] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.289271 13092 raft_consensus.cc:385] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.289556 13092 raft_consensus.cc:740] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.290108 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 2411a153e7194406a2bb7b879c08d7dd (DEFAULT_TABLE table=default.Table_72fe02818135414ca8fb8033ebda39c7 [id=02441df5c4e6405fa8a72aa72b16687e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.291146 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2411a153e7194406a2bb7b879c08d7dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.291630 13013 ts_tablet_manager.cc:1434] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.023s user 0.001s sys 0.003s
I20260926 19:58:40.292475 13013 tablet_bootstrap.cc:492] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:40.298107 13092 consensus_queue.cc:260] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.298849 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef062e1775bd4a3c859124eaaba81399" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:40.301268 12521 leader_election.cc:343] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:40.302551 12521 leader_election.cc:343] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.311674 13092 ts_tablet_manager.cc:1434] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260926 19:58:40.318809 13092 tablet_bootstrap.cc:492] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:40.327325 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_be1467ad5f7e4a92b7f91320650f32bd" } new_table_name: "default.Table_a34ea809f74841d18b133415daf8192a" modify_external_catalogs: true
I20260926 19:58:40.338582 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_693c5b4b4dc446378398965b02ddeea5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:40.348678 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 2411a153e7194406a2bb7b879c08d7dd (DEFAULT_TABLE table=default.Table_72fe02818135414ca8fb8033ebda39c7 [id=02441df5c4e6405fa8a72aa72b16687e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.349802 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2411a153e7194406a2bb7b879c08d7dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.351676 13092 tablet_bootstrap.cc:654] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.357750 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c46bbc68304f4ebb80a240478a92349d" } modify_external_catalogs: true
I20260926 19:58:40.358845 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c46bbc68304f4ebb80a240478a92349d" } modify_external_catalogs: true
I20260926 19:58:40.365067 13013 tablet_bootstrap.cc:654] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.367712 13203 ts_tablet_manager.cc:1929] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:40.368175 13203 log.cc:1199] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/6f1dafade40a46649ac2780b8ad3b9ff
I20260926 19:58:40.369072 13203 ts_tablet_manager.cc:1950] T 6f1dafade40a46649ac2780b8ad3b9ff P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
W20260926 19:58:40.371675 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_693c5b4b4dc446378398965b02ddeea5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:40.351495 12521 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.11.233.253:37913, user_credentials={real_user=slave}} blocked reactor thread for 44949.8us
I20260926 19:58:40.380070 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 6f1dafade40a46649ac2780b8ad3b9ff (table default.Table_b3e35b31063d4ba7a63fc5af3f5dd752 [id=e8e5454a4f9243988f48a9c7d26a7736]) successfully deleted
W20260926 19:58:40.382866 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 00767b01d2ad4ca5b6cfcc566739f61d because the tablet was not found. No further retry: Not found: Tablet not found: 00767b01d2ad4ca5b6cfcc566739f61d
I20260926 19:58:40.386034 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 0ea151df659944f9a094fe9410c3f4bb (DEFAULT_TABLE table=default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.387053 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ea151df659944f9a094fe9410c3f4bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.388366 13031 ts_tablet_manager.cc:1929] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:40.388800 13031 log.cc:1199] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/6f1dafade40a46649ac2780b8ad3b9ff
I20260926 19:58:40.389943 13203 tablet_replica.cc:333] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:40.390973 13203 raft_consensus.cc:2243] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:40.391506 13203 raft_consensus.cc:2272] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.398170 13092 tablet_bootstrap.cc:492] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:40.398581 13092 ts_tablet_manager.cc:1403] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.086s user 0.011s sys 0.006s
I20260926 19:58:40.401463 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 0ea151df659944f9a094fe9410c3f4bb (DEFAULT_TABLE table=default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.402366 13031 ts_tablet_manager.cc:1950] T 6f1dafade40a46649ac2780b8ad3b9ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:40.402709 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ea151df659944f9a094fe9410c3f4bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.405066 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 6f1dafade40a46649ac2780b8ad3b9ff (table default.Table_b3e35b31063d4ba7a63fc5af3f5dd752 [id=e8e5454a4f9243988f48a9c7d26a7736]) successfully deleted
I20260926 19:58:40.405800 13203 ts_tablet_manager.cc:1916] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:40.409636 12899 tablet_service.cc:1511] Processing CreateTablet for tablet e550463602ff4b5f98e10dd593897bcf (DEFAULT_TABLE table=default.Table_1ef063c29bb54a258c2650cf73e9abd0 [id=21d0c2efe78e438e8fad11d811c463ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.410681 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e550463602ff4b5f98e10dd593897bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.401355 13092 raft_consensus.cc:359] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.417529 12562 tablet_service.cc:1511] Processing CreateTablet for tablet e550463602ff4b5f98e10dd593897bcf (DEFAULT_TABLE table=default.Table_1ef063c29bb54a258c2650cf73e9abd0 [id=21d0c2efe78e438e8fad11d811c463ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.418005 13092 raft_consensus.cc:385] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.418269 13092 raft_consensus.cc:740] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.418650 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e550463602ff4b5f98e10dd593897bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.418923 13092 consensus_queue.cc:260] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.420198 13031 tablet_replica.cc:333] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:40.422376 13031 raft_consensus.cc:2243] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:40.427310 13031 raft_consensus.cc:2272] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.425132 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 732a34f4894940b3889bab5f0c0891e0 (DEFAULT_TABLE table=default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.431509 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732a34f4894940b3889bab5f0c0891e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.436683 12901 tablet_service.cc:1511] Processing CreateTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 (DEFAULT_TABLE table=default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95 [id=4692b08d69b344cdaf9226c62886da70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.437569 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (DEFAULT_TABLE table=default.Table_9ef822f7aa5c4261accca0ae1a201026 [id=dd5c0a6b728948278a5dd48f8ec0bf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.437822 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b19f42511ad14d8a88fc0eaefb82e335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.436904 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 732a34f4894940b3889bab5f0c0891e0 (DEFAULT_TABLE table=default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.438495 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.438773 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732a34f4894940b3889bab5f0c0891e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.439368 12564 tablet_service.cc:1511] Processing CreateTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 (DEFAULT_TABLE table=default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95 [id=4692b08d69b344cdaf9226c62886da70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.439810 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (DEFAULT_TABLE table=default.Table_9ef822f7aa5c4261accca0ae1a201026 [id=dd5c0a6b728948278a5dd48f8ec0bf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.440389 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b19f42511ad14d8a88fc0eaefb82e335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.441980 13031 ts_tablet_manager.cc:1916] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:40.420725 13092 ts_tablet_manager.cc:1434] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.022s user 0.004s sys 0.000s
I20260926 19:58:40.449249 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 28b592db99c1495aafb555ff2a76a154
I20260926 19:58:40.449692 13092 tablet_bootstrap.cc:492] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:40.450915 12393 catalog_manager.cc:6934] ReplaceTablet: tablet 28b592db99c1495aafb555ff2a76a154 deleted and replaced by tablet 0a7d5e580f5645ab95fa03d5f669dc32
I20260926 19:58:40.451071 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 28b592db99c1495aafb555ff2a76a154 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a7d5e580f5645ab95fa03d5f669dc32) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:40.452976 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet 28b592db99c1495aafb555ff2a76a154 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a7d5e580f5645ab95fa03d5f669dc32) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.455000 12393 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_353f84cf17724709af77fc8d316ca0c4" } new_table_name: "default.Table_cdfb152c5e784e0ab640ed6e68bd8bda" modify_external_catalogs: true
I20260926 19:58:40.473814 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.482113 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet ffac209a6cd8440289ee7765c3c24b40
I20260926 19:58:40.483659 12392 catalog_manager.cc:6934] ReplaceTablet: tablet ffac209a6cd8440289ee7765c3c24b40 deleted and replaced by tablet 189eaca5fb2b4b0d83a2873e0495e5ff
I20260926 19:58:40.486279 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet ffac209a6cd8440289ee7765c3c24b40 with delete_type TABLET_DATA_DELETED (replaced by tablet 189eaca5fb2b4b0d83a2873e0495e5ff) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.487519 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet ffac209a6cd8440289ee7765c3c24b40 with delete_type TABLET_DATA_DELETED (replaced by tablet 189eaca5fb2b4b0d83a2873e0495e5ff) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:40.493453 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_199fcdc45b23492aacc9631f50874ddb" } modify_external_catalogs: true
I20260926 19:58:40.494133 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_199fcdc45b23492aacc9631f50874ddb" } modify_external_catalogs: true
I20260926 19:58:40.499758 13173 raft_consensus.cc:493] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.500221 13173 raft_consensus.cc:515] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.502116 13173 leader_election.cc:290] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:40.502477 13203 ts_tablet_manager.cc:1929] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:40.502954 13203 log.cc:1199] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/e365004bba0d428cbb1900ad558efe5e
I20260926 19:58:40.504256 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d75be43e841e7abcc9e9db800aa2d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:40.506579 12521 leader_election.cc:343] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.510526 13013 tablet_bootstrap.cc:492] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:40.510922 13013 ts_tablet_manager.cc:1403] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.219s user 0.017s sys 0.007s
I20260926 19:58:40.516799 12735 tablet_service.cc:1558] Processing DeleteTablet for tablet 4eca1e5734324f99b9362915bdd00239 with delete_type TABLET_DATA_DELETED (replaced by tablet b19f42511ad14d8a88fc0eaefb82e335) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:40.517999 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 99cf26cf4838454c98aa8337341ffd06 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ea151df659944f9a094fe9410c3f4bb) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:40.518015 12736 tablet_service.cc:1558] Processing DeleteTablet for tablet ef062e1775bd4a3c859124eaaba81399 with delete_type TABLET_DATA_DELETED (replaced by tablet da8875d46f7c46819ee67e650b098c27) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:40.518728 13092 tablet_bootstrap.cc:654] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.519594 12738 tablet_service.cc:1511] Processing CreateTablet for tablet ed45552637024bec99a76da079f352ec (DEFAULT_TABLE table=default.Table_dd0aa5f5971b4394a8aa99f77fa7d0eb [id=667dce3292c94f53b899dcba291d56a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.520987 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed45552637024bec99a76da079f352ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.530683 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 2411a153e7194406a2bb7b879c08d7dd (DEFAULT_TABLE table=default.Table_72fe02818135414ca8fb8033ebda39c7 [id=02441df5c4e6405fa8a72aa72b16687e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.531868 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2411a153e7194406a2bb7b879c08d7dd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:40.539204 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba3f3931a71048fd874c1131d44f7b00 consensus state has no leader
I20260926 19:58:40.542879 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ea151df659944f9a094fe9410c3f4bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:40.543394 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 0ea151df659944f9a094fe9410c3f4bb (DEFAULT_TABLE table=default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.544667 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ea151df659944f9a094fe9410c3f4bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.574999 12741 tablet_service.cc:1511] Processing CreateTablet for tablet e550463602ff4b5f98e10dd593897bcf (DEFAULT_TABLE table=default.Table_1ef063c29bb54a258c2650cf73e9abd0 [id=21d0c2efe78e438e8fad11d811c463ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.576026 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e550463602ff4b5f98e10dd593897bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.582716 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 732a34f4894940b3889bab5f0c0891e0 (DEFAULT_TABLE table=default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.582996 12737 tablet_service.cc:1511] Processing CreateTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 (DEFAULT_TABLE table=default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95 [id=4692b08d69b344cdaf9226c62886da70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.583849 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 732a34f4894940b3889bab5f0c0891e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.584393 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b19f42511ad14d8a88fc0eaefb82e335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.597826 13092 tablet_bootstrap.cc:492] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:40.599872 13092 ts_tablet_manager.cc:1403] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.150s user 0.010s sys 0.000s
I20260926 19:58:40.601920 13149 raft_consensus.cc:493] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.602536 13149 raft_consensus.cc:515] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.604717 13149 leader_election.cc:290] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:40.604538 13048 raft_consensus.cc:493] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.610679 13048 raft_consensus.cc:515] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.612604 13048 leader_election.cc:290] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:40.606822 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "381320b4d2844e2289a5ae390e6b0503" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:40.614902 12393 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 189eaca5fb2b4b0d83a2873e0495e5ff has no consensus state
W20260926 19:58:40.615157 12696 leader_election.cc:343] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.617378 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38d785b63734ddd9f2e4f47ef55c7c9" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:40.524312 13203 ts_tablet_manager.cc:1950] T e365004bba0d428cbb1900ad558efe5e P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:40.622506 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d38d785b63734ddd9f2e4f47ef55c7c9" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:40.623255 12595 raft_consensus.cc:2393] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:40.627660 13032 ts_tablet_manager.cc:1929] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:40.629712 13092 raft_consensus.cc:359] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.630357 13092 raft_consensus.cc:385] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.624704 13128 raft_consensus.cc:493] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.630612 13092 raft_consensus.cc:740] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.634765 13092 consensus_queue.cc:260] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.635228 13013 raft_consensus.cc:359] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.635859 13013 raft_consensus.cc:385] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.636097 13013 raft_consensus.cc:740] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.636765 13092 ts_tablet_manager.cc:1434] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260926 19:58:40.631283 13173 raft_consensus.cc:493] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:40.637563 12382 consensus_peers.cc:597] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 -> Peer db2490adb6e84090b780f52470e580ab (127.11.233.253:37913): Couldn't send request to peer db2490adb6e84090b780f52470e580ab. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260926 19:58:40.639719 13013 consensus_queue.cc:260] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.642521 12858 leader_election.cc:304] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:40.643724 13048 raft_consensus.cc:3060] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.628422 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet 28b592db99c1495aafb555ff2a76a154 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a7d5e580f5645ab95fa03d5f669dc32) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:40.644361 13032 log.cc:1199] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/00767b01d2ad4ca5b6cfcc566739f61d
I20260926 19:58:40.645231 13032 ts_tablet_manager.cc:1950] T 00767b01d2ad4ca5b6cfcc566739f61d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:40.647470 13032 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 56) took 1355 ms. Trace:
I20260926 19:58:40.634280 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "381320b4d2844e2289a5ae390e6b0503" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:40.647858 13032 rpcz_store.cc:276] 0926 19:58:39.291643 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:39.291758 (+ 115us) service_pool.cc:224] Handling call
0926 19:58:40.180003 (+888245us) ts_tablet_manager.cc:1151] Deleting tablet 00767b01d2ad4ca5b6cfcc566739f61d
0926 19:58:40.180056 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:40.565757 (+385701us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:40.627446 (+ 61689us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:40.647385 (+ 19939us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":887707}
I20260926 19:58:40.648993 13032 tablet_replica.cc:333] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:40.652701 13048 raft_consensus.cc:2749] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:40.654958 12735 tablet_service.cc:1558] Processing DeleteTablet for tablet ffac209a6cd8440289ee7765c3c24b40 with delete_type TABLET_DATA_DELETED (replaced by tablet 189eaca5fb2b4b0d83a2873e0495e5ff) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:40.658674 12735 tablet_service.cc:1511] Processing CreateTablet for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (DEFAULT_TABLE table=default.Table_9ef822f7aa5c4261accca0ae1a201026 [id=dd5c0a6b728948278a5dd48f8ec0bf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.660149 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.641681 13173 raft_consensus.cc:515] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.641099 13128 raft_consensus.cc:515] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.665834 13032 raft_consensus.cc:2243] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 19:58:40.666090 12695 leader_election.cc:343] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.666431 13032 raft_consensus.cc:2272] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.666610 12695 leader_election.cc:304] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:40.674620 13203 tablet_replica.cc:333] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:40.675469 13203 raft_consensus.cc:2243] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:40.675927 13203 raft_consensus.cc:2272] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.677320 13180 raft_consensus.cc:2749] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:40.677579 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet e365004bba0d428cbb1900ad558efe5e (table default.Table_9ac55a637c1e41f0bfca50930bd701c0 [id=47b086c8685e4ac19c14784f1691cae1]) successfully deleted
W20260926 19:58:40.677318 13012 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.578s user 0.023s sys 0.033s
W20260926 19:58:40.678889 13012 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.580s user 0.023s sys 0.033s
I20260926 19:58:40.682687 13128 leader_election.cc:290] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:40.687366 13092 tablet_bootstrap.cc:492] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:40.689317 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6d07cb4c954f23be80b315a5748adc" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:40.692934 13092 tablet_bootstrap.cc:654] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.692766 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_950e6b311c89471aa40a33da527b1110"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:40.694010 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_29ce86fb5af04ec4b8411a4f3135b79d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:40.696946 13012 tablet_bootstrap.cc:492] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
W20260926 19:58:40.697347 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 0ea151df659944f9a094fe9410c3f4bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ea151df659944f9a094fe9410c3f4bb already in progress: creating tablet
I20260926 19:58:40.700984 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_5aefddf9a623467fbcae8313cbfe329d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:40.702126 12521 leader_election.cc:343] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.697499 13012 ts_tablet_manager.cc:1403] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.664s user 0.029s sys 0.036s
I20260926 19:58:40.700876 13032 ts_tablet_manager.cc:1916] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:40.707410 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c683dc1272f4290a56f7fc7010ba038" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:40.708000 12760 raft_consensus.cc:3060] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.708513 13173 leader_election.cc:290] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:40.710866 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_be184ee4abbf48999888ee786db3d2e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:40.712414 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9acbd4e5aa23433d860d399d78485d0d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:40.714231 12521 leader_election.cc:343] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.722445 13012 raft_consensus.cc:359] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.723268 13012 raft_consensus.cc:385] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.723569 13012 raft_consensus.cc:740] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.724505 13012 consensus_queue.cc:260] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.725272 13203 ts_tablet_manager.cc:1916] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:40.715737 12760 raft_consensus.cc:2468] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:40.741123 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d75be43e841e7abcc9e9db800aa2d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:40.741750 12761 raft_consensus.cc:2468] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:40.744323 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6d07cb4c954f23be80b315a5748adc" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:40.744434 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_124ad4c229ac49ccbc0243da96985a9e" } new_table_name: "default.Table_924b11c341f3451e8049f2e8dc462bfa" modify_external_catalogs: true
I20260926 19:58:40.750349 13048 raft_consensus.cc:493] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.750912 13048 raft_consensus.cc:515] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:40.753262 13048 leader_election.cc:290] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:58:40.754920 12858 leader_election.cc:343] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 4eca1e5734324f99b9362915bdd00239
W20260926 19:58:40.756610 12857 leader_election.cc:343] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 4eca1e5734324f99b9362915bdd00239
I20260926 19:58:40.756969 12857 leader_election.cc:304] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:40.765655 13048 raft_consensus.cc:2749] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:40.767311 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 27ccbbe1a5784e97827a31e9febd8c20 has no consensus state
I20260926 19:58:40.774013 13180 consensus_queue.cc:1048] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:40.774402 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9acbd4e5aa23433d860d399d78485d0d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:40.774976 12761 raft_consensus.cc:2468] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:40.775978 12391 master_service.cc:960] ReplaceTablet: received request to replace tablet 0ea151df659944f9a094fe9410c3f4bb from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:40.777594 13012 ts_tablet_manager.cc:1434] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.072s user 0.007s sys 0.000s
I20260926 19:58:40.779539 13012 tablet_bootstrap.cc:492] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:40.781864 12391 master_service.cc:960] ReplaceTablet: received request to replace tablet 1d25f740c1334c5cb53d7cffa3eb275e from {username='slave'} at 127.0.0.1:45734
W20260926 19:58:40.783250 12696 consensus_peers.cc:597] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:40.787035 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 00767b01d2ad4ca5b6cfcc566739f61d (table default.Table_d2d87877a7c4476aa7e41623c09ed4df [id=d4475cbaf3a74d19818f03cdb5ca6537]) successfully deleted
W20260926 19:58:40.788189 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be184ee4abbf48999888ee786db3d2e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:40.788779 12519 leader_election.cc:304] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.790257 13173 raft_consensus.cc:2804] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
W20260926 19:58:40.788714 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29ce86fb5af04ec4b8411a4f3135b79d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:40.789301 12353 master_runner.cc:187] Error getting master registration for 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111),
I20260926 19:58:40.790755 13173 raft_consensus.cc:697] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:40.790958 12353 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 19:58:40.789484 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_950e6b311c89471aa40a33da527b1110 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:40.791528 13173 consensus_queue.cc:237] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:40.794229 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5aefddf9a623467fbcae8313cbfe329d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:40.795475 13092 tablet_bootstrap.cc:492] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:40.795951 13092 ts_tablet_manager.cc:1403] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.109s user 0.026s sys 0.009s
I20260926 19:58:40.798377 13092 raft_consensus.cc:359] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.804011 13013 ts_tablet_manager.cc:1434] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.293s user 0.014s sys 0.005s
I20260926 19:58:40.804373 13092 raft_consensus.cc:385] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.804433 12519 leader_election.cc:304] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.804929 13013 tablet_bootstrap.cc:492] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:40.805197 13092 raft_consensus.cc:740] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.805583 13128 raft_consensus.cc:2804] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:40.805948 13128 raft_consensus.cc:493] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:40.806244 13128 raft_consensus.cc:3060] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.806723 13092 consensus_queue.cc:260] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.809032 13203 ts_tablet_manager.cc:1929] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:40.809394 13203 log.cc:1199] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/28b592db99c1495aafb555ff2a76a154
I20260926 19:58:40.819579 13203 ts_tablet_manager.cc:1950] T 28b592db99c1495aafb555ff2a76a154 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:40.822330 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 28b592db99c1495aafb555ff2a76a154 (table default.Table_5e21794a58a648c8a5806d4c535a8f37 [id=c2c079dc87ec480a82aa5b57b110da17]) successfully deleted
I20260926 19:58:40.822860 13180 raft_consensus.cc:493] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.823318 13180 raft_consensus.cc:515] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.824024 13092 ts_tablet_manager.cc:1434] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260926 19:58:40.824867 13092 tablet_bootstrap.cc:492] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:40.829197 13013 tablet_bootstrap.cc:654] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.830201 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25f740c1334c5cb53d7cffa3eb275e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:40.831421 13180 leader_election.cc:290] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:58:40.833267 12696 leader_election.cc:343] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.834167 13031 ts_tablet_manager.cc:1929] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:40.834524 13031 log.cc:1199] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/00767b01d2ad4ca5b6cfcc566739f61d
W20260926 19:58:40.836268 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet ffac209a6cd8440289ee7765c3c24b40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffac209a6cd8440289ee7765c3c24b40 already in progress: creating tablet
I20260926 19:58:40.836599 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25f740c1334c5cb53d7cffa3eb275e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
W20260926 19:58:40.837975 12695 leader_election.cc:343] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.838265 13092 tablet_bootstrap.cc:654] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:40.838366 12695 leader_election.cc:304] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:40.839783 13180 raft_consensus.cc:2749] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:40.841570 13128 raft_consensus.cc:515] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.844491 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_924b11c341f3451e8049f2e8dc462bfa" } new_table_name: "default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1" modify_external_catalogs: true
I20260926 19:58:40.861681 13128 leader_election.cc:290] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:40.861652 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d75be43e841e7abcc9e9db800aa2d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:40.863046 12521 leader_election.cc:343] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:40.866088 12519 leader_election.cc:343] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.866549 12519 leader_election.cc:304] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.868046 12519 leader_election.cc:304] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.869110 13212 raft_consensus.cc:2749] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:40.869908 13212 raft_consensus.cc:2804] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:40.870241 13212 raft_consensus.cc:493] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:40.870541 13212 raft_consensus.cc:3060] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.870813 13180 raft_consensus.cc:493] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:40.871328 13180 raft_consensus.cc:515] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.874442 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf4547f148f49b8943f4e1c7f2132a7" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:40.876147 12696 leader_election.cc:343] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.877696 13180 leader_election.cc:290] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:40.878245 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf4547f148f49b8943f4e1c7f2132a7" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:40.878832 12595 raft_consensus.cc:2468] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:40.880131 12695 leader_election.cc:304] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.881274 13180 raft_consensus.cc:2804] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:40.881704 13180 raft_consensus.cc:493] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:40.882090 13180 raft_consensus.cc:3060] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.883067 13212 raft_consensus.cc:515] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.885857 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9acbd4e5aa23433d860d399d78485d0d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:40.887200 12521 leader_election.cc:343] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:40.893998 13212 leader_election.cc:290] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:40.893726 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 27ccbbe1a5784e97827a31e9febd8c20 has no consensus state
I20260926 19:58:40.904590 13149 consensus_queue.cc:1048] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:40.908100 12519 leader_election.cc:343] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: ef062e1775bd4a3c859124eaaba81399
I20260926 19:58:40.908528 12519 leader_election.cc:304] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.914047 13212 raft_consensus.cc:2749] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:40.915264 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d75be43e841e7abcc9e9db800aa2d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:40.916538 12760 raft_consensus.cc:3060] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.922451 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9acbd4e5aa23433d860d399d78485d0d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:40.923198 12761 raft_consensus.cc:3060] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:40.835122 13031 ts_tablet_manager.cc:1950] T 00767b01d2ad4ca5b6cfcc566739f61d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:40.927716 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 00767b01d2ad4ca5b6cfcc566739f61d (table default.Table_d2d87877a7c4476aa7e41623c09ed4df [id=d4475cbaf3a74d19818f03cdb5ca6537]) successfully deleted
I20260926 19:58:40.930698 12761 raft_consensus.cc:2468] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:40.932164 12519 leader_election.cc:304] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:40.933681 13212 raft_consensus.cc:2804] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:40.934149 13212 raft_consensus.cc:697] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:40.933972 13180 raft_consensus.cc:515] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.934892 13212 consensus_queue.cc:237] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:40.942873 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 0ea151df659944f9a094fe9410c3f4bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ea151df659944f9a094fe9410c3f4bb already in progress: creating tablet
I20260926 19:58:40.946260 13092 tablet_bootstrap.cc:492] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:40.945608 12388 catalog_manager.cc:5838] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:40.946676 13092 ts_tablet_manager.cc:1403] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.122s user 0.012s sys 0.002s
I20260926 19:58:40.946852 12388 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet b46a5297ef854afd8ef783bf0692135e (table default.Table_1281608b01e84b90b871e71e1adc84b9 [id=a774283438c64c668a5b2e9e4d62ed99]). Expected version 1 got 0
I20260926 19:58:40.947679 12388 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet b82d75be43e841e7abcc9e9db800aa2d (table default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa]). Expected version 2 got 0
I20260926 19:58:40.948561 12388 catalog_manager.cc:5838] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:40.949188 13092 raft_consensus.cc:359] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.949822 13092 raft_consensus.cc:385] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:40.949910 12388 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_1305c705a9be4c02863b9f3d5d014850 [id=b1955ec80c98477c95b3b06cb73f3206]). Expected version 2 got 0
I20260926 19:58:40.950084 13092 raft_consensus.cc:740] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:40.951051 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet d38d785b63734ddd9f2e4f47ef55c7c9
I20260926 19:58:40.950760 13092 consensus_queue.cc:260] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:40.953675 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet d38d785b63734ddd9f2e4f47ef55c7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:40 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.954394 13203 tablet_replica.cc:333] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:40.955088 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet d38d785b63734ddd9f2e4f47ef55c7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:40 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:40.955191 13203 raft_consensus.cc:2243] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:40.956094 13203 raft_consensus.cc:2272] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.958231 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 8c683dc1272f4290a56f7fc7010ba038
I20260926 19:58:40.959267 13092 ts_tablet_manager.cc:1434] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260926 19:58:40.960139 13092 tablet_bootstrap.cc:492] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:40.963529 13031 tablet_replica.cc:333] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:40.964191 13031 raft_consensus.cc:2243] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:40.964599 13031 raft_consensus.cc:2272] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:40.966185 13031 ts_tablet_manager.cc:1916] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:40.973044 12562 tablet_service.cc:1511] Processing CreateTablet for tablet fb593ce2cd4147febebfe442de9fdd28 (DEFAULT_TABLE table=default.Table_693c5b4b4dc446378398965b02ddeea5 [id=d088bdbe9cf34c41adfc15548ca5521a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.973654 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c683dc1272f4290a56f7fc7010ba038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:40 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.974162 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb593ce2cd4147febebfe442de9fdd28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.974242 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet ef062e1775bd4a3c859124eaaba81399 with delete_type TABLET_DATA_DELETED (replaced by tablet da8875d46f7c46819ee67e650b098c27) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.974817 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f1dafade40a46649ac2780b8ad3b9ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.976543 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 (DEFAULT_TABLE table=default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.977689 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3215bbd5be56438491a73925ac9ccbe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.978282 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 64149f9e4d384ddbb922291ac7de35b7 (DEFAULT_TABLE table=default.Table_5b2c3bb01e984005a49cb679fa197189 [id=00773dee014a47fca49e7dfd9ed0d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.979252 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64149f9e4d384ddbb922291ac7de35b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.980087 12565 tablet_service.cc:1558] Processing DeleteTablet for tablet 99cf26cf4838454c98aa8337341ffd06 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ea151df659944f9a094fe9410c3f4bb) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:40.985165 13013 tablet_bootstrap.cc:492] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:40.985579 13013 ts_tablet_manager.cc:1403] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.181s user 0.010s sys 0.000s
I20260926 19:58:40.986690 12899 tablet_service.cc:1511] Processing CreateTablet for tablet fb593ce2cd4147febebfe442de9fdd28 (DEFAULT_TABLE table=default.Table_693c5b4b4dc446378398965b02ddeea5 [id=d088bdbe9cf34c41adfc15548ca5521a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.987841 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 (DEFAULT_TABLE table=default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.988080 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb593ce2cd4147febebfe442de9fdd28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.986711 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c683dc1272f4290a56f7fc7010ba038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:40 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:40.988735 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3215bbd5be56438491a73925ac9ccbe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.986826 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 64149f9e4d384ddbb922291ac7de35b7 (DEFAULT_TABLE table=default.Table_5b2c3bb01e984005a49cb679fa197189 [id=00773dee014a47fca49e7dfd9ed0d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:40.992403 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64149f9e4d384ddbb922291ac7de35b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:40.995087 13203 ts_tablet_manager.cc:1916] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:41.004773 12391 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet b46a5297ef854afd8ef783bf0692135e (table default.Table_1281608b01e84b90b871e71e1adc84b9 [id=a774283438c64c668a5b2e9e4d62ed99]). Expected version 1 got 0
I20260926 19:58:41.010766 12901 tablet_service.cc:1558] Processing DeleteTablet for tablet ef062e1775bd4a3c859124eaaba81399 with delete_type TABLET_DATA_DELETED (replaced by tablet da8875d46f7c46819ee67e650b098c27) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:41.010200 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9ef822f7aa5c4261accca0ae1a201026" } new_table_name: "default.Table_3152a51e41624d969eea96ab90354999" modify_external_catalogs: true
I20260926 19:58:41.011543 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 99cf26cf4838454c98aa8337341ffd06 with delete_type TABLET_DATA_DELETED (replaced by tablet 0ea151df659944f9a094fe9410c3f4bb) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:41.014585 13048 raft_consensus.cc:493] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:41.015236 13048 raft_consensus.cc:515] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.019402 13048 leader_election.cc:290] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:40.954222 12735 tablet_service.cc:1558] Processing DeleteTablet for tablet d38d785b63734ddd9f2e4f47ef55c7c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:40 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:41.027124 13032 ts_tablet_manager.cc:1929] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:41.027875 13032 log.cc:1199] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/6f1dafade40a46649ac2780b8ad3b9ff
I20260926 19:58:41.029268 13032 ts_tablet_manager.cc:1950] T 6f1dafade40a46649ac2780b8ad3b9ff P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:40.954643 12760 raft_consensus.cc:2468] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
W20260926 19:58:41.034983 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62e75eb4b3e8493b9d599c1c4ad4c67f consensus state has no leader
I20260926 19:58:41.043262 12389 master_service.cc:960] ReplaceTablet: received request to replace tablet 1d25f740c1334c5cb53d7cffa3eb275e from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:41.049329 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_5f027d31993e4d35a2937f694c90a59c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:41.053382 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 53668c46bd3441ad936bf990b3192420 (DEFAULT_TABLE table=default.Table_2b4f759d68744ea8b7d986616f8618d1 [id=faf55c4cd53c42a9b894d47f5064d220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.054623 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53668c46bd3441ad936bf990b3192420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.062887 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_d7b3b0491c1a417ea7d465c4662e7dee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:41.064545 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7b3b0491c1a417ea7d465c4662e7dee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.065940 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a958288944144fbd9cc934add059cdb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:41.067754 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a958288944144fbd9cc934add059cdb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.071282 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 1455b99a65ff44ff933825196708f485 (DEFAULT_TABLE table=default.Table_f1e412eb951f446990776744666978fc [id=bfb91cbd91534b4699fd4423b9923881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.072243 13013 raft_consensus.cc:359] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.071908 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 (DEFAULT_TABLE table=default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1 [id=36e060a7ae90436d8c3917819816609c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.072849 13013 raft_consensus.cc:385] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:41.073074 13013 raft_consensus.cc:740] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:41.073609 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ccbbe1a5784e97827a31e9febd8c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.076215 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 53668c46bd3441ad936bf990b3192420 (DEFAULT_TABLE table=default.Table_2b4f759d68744ea8b7d986616f8618d1 [id=faf55c4cd53c42a9b894d47f5064d220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.077258 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53668c46bd3441ad936bf990b3192420. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:41.082618 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f027d31993e4d35a2937f694c90a59c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.082886 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 (DEFAULT_TABLE table=default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1 [id=36e060a7ae90436d8c3917819816609c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.083858 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ccbbe1a5784e97827a31e9febd8c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.083619 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 1455b99a65ff44ff933825196708f485 (DEFAULT_TABLE table=default.Table_f1e412eb951f446990776744666978fc [id=bfb91cbd91534b4699fd4423b9923881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.084613 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1455b99a65ff44ff933825196708f485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.090451 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b2cab18b22ac46d9bd64e47ac24851e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:41.091986 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2cab18b22ac46d9bd64e47ac24851e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.092228 13092 tablet_bootstrap.cc:654] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:41.044239 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c683dc1272f4290a56f7fc7010ba038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:40 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:41.098934 12394 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5b2c3bb01e984005a49cb679fa197189" } new_table_name: "default.Table_499a09d79630415996bc999c678ae00f" modify_external_catalogs: true
I20260926 19:58:41.100631 13032 tablet_replica.cc:333] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:41.103415 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 (DEFAULT_TABLE table=default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.103775 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:41.104424 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3215bbd5be56438491a73925ac9ccbe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.104898 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 53668c46bd3441ad936bf990b3192420 (DEFAULT_TABLE table=default.Table_2b4f759d68744ea8b7d986616f8618d1 [id=faf55c4cd53c42a9b894d47f5064d220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.106433 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 6f1dafade40a46649ac2780b8ad3b9ff (table default.Table_b3e35b31063d4ba7a63fc5af3f5dd752 [id=e8e5454a4f9243988f48a9c7d26a7736]) successfully deleted
W20260926 19:58:41.107112 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 4eca1e5734324f99b9362915bdd00239 because the tablet was not found. No further retry: Not found: Tablet not found: 4eca1e5734324f99b9362915bdd00239
W20260926 19:58:41.107702 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 99cf26cf4838454c98aa8337341ffd06 because the tablet was not found. No further retry: Not found: Tablet not found: 99cf26cf4838454c98aa8337341ffd06
W20260926 19:58:41.108281 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet ef062e1775bd4a3c859124eaaba81399 because the tablet was not found. No further retry: Not found: Tablet not found: ef062e1775bd4a3c859124eaaba81399
W20260926 19:58:41.108865 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 0ea151df659944f9a094fe9410c3f4bb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ea151df659944f9a094fe9410c3f4bb already in progress: creating tablet
I20260926 19:58:41.117698 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c683dc1272f4290a56f7fc7010ba038" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:41.119781 12735 tablet_service.cc:1511] Processing CreateTablet for tablet 64149f9e4d384ddbb922291ac7de35b7 (DEFAULT_TABLE table=default.Table_5b2c3bb01e984005a49cb679fa197189 [id=00773dee014a47fca49e7dfd9ed0d584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.120369 13012 tablet_bootstrap.cc:654] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:41.122272 12519 leader_election.cc:304] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:41.120908 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64149f9e4d384ddbb922291ac7de35b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.130945 13032 raft_consensus.cc:2243] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:41.130807 12737 tablet_service.cc:1511] Processing CreateTablet for tablet fb593ce2cd4147febebfe442de9fdd28 (DEFAULT_TABLE table=default.Table_693c5b4b4dc446378398965b02ddeea5 [id=d088bdbe9cf34c41adfc15548ca5521a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.131404 13032 raft_consensus.cc:2272] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.134276 12734 tablet_service.cc:1511] Processing CreateTablet for tablet 1455b99a65ff44ff933825196708f485 (DEFAULT_TABLE table=default.Table_f1e412eb951f446990776744666978fc [id=bfb91cbd91534b4699fd4423b9923881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.135422 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53668c46bd3441ad936bf990b3192420. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:41.109432 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 28b592db99c1495aafb555ff2a76a154 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28b592db99c1495aafb555ff2a76a154 already in progress: creating tablet
I20260926 19:58:41.139175 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb593ce2cd4147febebfe442de9fdd28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.140393 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c683dc1272f4290a56f7fc7010ba038" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:41.141103 12760 raft_consensus.cc:2393] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:41.141395 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf4547f148f49b8943f4e1c7f2132a7" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:41.142789 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1455b99a65ff44ff933825196708f485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.152671 13032 ts_tablet_manager.cc:1916] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:41.154798 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_3152a51e41624d969eea96ab90354999 [id=dd5c0a6b728948278a5dd48f8ec0bf6f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 62e75eb4b3e8493b9d599c1c4ad4c67f consensus state has no leader
I20260926 19:58:41.154146 13223 consensus_queue.cc:1048] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:41.157855 12696 leader_election.cc:343] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:41.166785 13048 raft_consensus.cc:493] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:41.167272 13048 raft_consensus.cc:515] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.169224 13048 leader_election.cc:290] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:41.169745 13180 leader_election.cc:290] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:58:41.170753 12394 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 64149f9e4d384ddbb922291ac7de35b7 consensus state has no leader
W20260926 19:58:41.172115 12857 leader_election.cc:343] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: ef062e1775bd4a3c859124eaaba81399
I20260926 19:58:41.172955 13180 consensus_queue.cc:1048] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:41.178594 12391 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_950e6b311c89471aa40a33da527b1110" } modify_external_catalogs: true
I20260926 19:58:41.179159 12391 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_950e6b311c89471aa40a33da527b1110" } modify_external_catalogs: true
I20260926 19:58:41.186136 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1455b99a65ff44ff933825196708f485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.187641 12387 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet b46a5297ef854afd8ef783bf0692135e (table default.Table_1281608b01e84b90b871e71e1adc84b9 [id=a774283438c64c668a5b2e9e4d62ed99]). Expected version 1 got 0
I20260926 19:58:41.190227 12388 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95" } modify_external_catalogs: true
I20260926 19:58:41.190728 12388 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95" } modify_external_catalogs: true
I20260926 19:58:41.197942 12389 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet b46a5297ef854afd8ef783bf0692135e (table default.Table_1281608b01e84b90b871e71e1adc84b9 [id=a774283438c64c668a5b2e9e4d62ed99]). Expected version 1 got 0
I20260926 19:58:41.207212 13212 raft_consensus.cc:2804] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:41.207715 13212 raft_consensus.cc:697] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:41.208496 13212 consensus_queue.cc:237] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:41.215149 12695 consensus_peers.cc:597] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:41.215946 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 (DEFAULT_TABLE table=default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1 [id=36e060a7ae90436d8c3917819816609c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.217901 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ccbbe1a5784e97827a31e9febd8c20. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:41.217854 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 95676.4us
I20260926 19:58:41.218773 12858 leader_election.cc:304] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:41.221055 13048 raft_consensus.cc:3060] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.220021 12391 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet f2148d52b9e6446da3dd1a8b630ead9d
I20260926 19:58:41.226320 12902 tablet_service.cc:1511] Processing CreateTablet for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.227411 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd92a1a62ab24647bef7c9eefdf9c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.228580 13048 raft_consensus.cc:2749] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:41.229774 13048 raft_consensus.cc:493] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:41.230584 13048 raft_consensus.cc:515] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.238936 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_cdfb152c5e784e0ab640ed6e68bd8bda" } new_table_name: "default.Table_df863789c3ff4a31a445becba51537cf" modify_external_catalogs: true
I20260926 19:58:41.250005 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet b19f42511ad14d8a88fc0eaefb82e335
I20260926 19:58:41.220711 12389 catalog_manager.cc:5838] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:41.252640 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:41.233489 12565 tablet_service.cc:1511] Processing CreateTablet for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.275321 13012 tablet_bootstrap.cc:492] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:41.275761 13012 ts_tablet_manager.cc:1403] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.496s user 0.015s sys 0.000s
I20260926 19:58:41.278266 12739 tablet_service.cc:1511] Processing CreateTablet for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.187403 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf4547f148f49b8943f4e1c7f2132a7" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:41.278559 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c (DEFAULT_TABLE table=default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.279076 12595 raft_consensus.cc:3060] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.279503 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd92a1a62ab24647bef7c9eefdf9c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.279595 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd184e6295d47aeba809be6aab9ce1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.264374 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99cf26cf4838454c98aa8337341ffd06" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:41.281921 12593 raft_consensus.cc:2468] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:41.256827 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:41.287515 12595 raft_consensus.cc:2468] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
W20260926 19:58:41.288708 12387 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 189eaca5fb2b4b0d83a2873e0495e5ff consensus state has no leader
I20260926 19:58:41.289909 12695 leader_election.cc:304] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:41.290701 13180 raft_consensus.cc:2804] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:41.293047 13180 raft_consensus.cc:697] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:41.246721 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef062e1775bd4a3c859124eaaba81399" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:41.293828 12594 raft_consensus.cc:2468] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:41.294588 13180 consensus_queue.cc:237] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.296676 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd92a1a62ab24647bef7c9eefdf9c9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.300038 13032 ts_tablet_manager.cc:1929] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:41.300520 13032 log.cc:1199] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/ffac209a6cd8440289ee7765c3c24b40
I20260926 19:58:41.301141 13032 ts_tablet_manager.cc:1950] T ffac209a6cd8440289ee7765c3c24b40 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:41.303656 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet ffac209a6cd8440289ee7765c3c24b40 (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]) successfully deleted
W20260926 19:58:41.307334 12857 leader_election.cc:343] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 99cf26cf4838454c98aa8337341ffd06
I20260926 19:58:41.312228 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:41.313222 13048 leader_election.cc:290] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:41.319532 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_4459dc1bf4e04cf6841f30e912b951ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:41.321198 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4459dc1bf4e04cf6841f30e912b951ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.324294 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c (DEFAULT_TABLE table=default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.325397 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd184e6295d47aeba809be6aab9ce1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.338320 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c (DEFAULT_TABLE table=default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.339389 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd184e6295d47aeba809be6aab9ce1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.339474 12858 leader_election.cc:304] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:41.341013 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (DEFAULT_TABLE table=default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.341861 12858 leader_election.cc:304] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:41.342396 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (DEFAULT_TABLE table=default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.343479 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189eaca5fb2b4b0d83a2873e0495e5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.343884 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189eaca5fb2b4b0d83a2873e0495e5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.343796 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 99465590a0e64b35b8a8f07e8777b524 (DEFAULT_TABLE table=default.Table_0c83b5f2aca049df9f95b55670a04502 [id=03889de8878549b7814b02a32caf20c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.344708 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99465590a0e64b35b8a8f07e8777b524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.349714 13032 tablet_replica.cc:333] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:41.350469 13032 raft_consensus.cc:2243] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:41.351786 13048 raft_consensus.cc:2804] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:41.352290 13048 raft_consensus.cc:493] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:41.352639 13048 raft_consensus.cc:3060] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.356776 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (DEFAULT_TABLE table=default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.358597 13231 raft_consensus.cc:2804] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:41.359109 13231 raft_consensus.cc:493] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:41.359556 13231 raft_consensus.cc:3060] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.365875 13013 consensus_queue.cc:260] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.367110 13032 raft_consensus.cc:2272] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.368072 13013 ts_tablet_manager.cc:1434] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.344s user 0.006s sys 0.000s
I20260926 19:58:41.369091 12387 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_1305c705a9be4c02863b9f3d5d014850 [id=b1955ec80c98477c95b3b06cb73f3206]). Expected version 2 got 0
I20260926 19:58:41.374622 13032 ts_tablet_manager.cc:1916] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:41.162750 13203 ts_tablet_manager.cc:1929] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:41.375219 13203 log.cc:1199] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/d38d785b63734ddd9f2e4f47ef55c7c9
I20260926 19:58:41.376621 13012 raft_consensus.cc:359] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.377259 13012 raft_consensus.cc:385] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:41.377527 13012 raft_consensus.cc:740] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:41.378319 13012 consensus_queue.cc:260] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.373917 12394 master_service.cc:960] ReplaceTablet: received request to replace tablet e550463602ff4b5f98e10dd593897bcf from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:41.374935 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_cc636537cf0c4914bb56b0c214adbf55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:41.383455 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc636537cf0c4914bb56b0c214adbf55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.384297 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_35ed1349ab724c9182fdb237e2630b50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:41.376351 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_170a499a114147fc9650b978b327f7e8" } new_table_name: "default.Table_e0ad88ad494c42759968cb1dc8501282" modify_external_catalogs: true
W20260926 19:58:41.385987 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35ed1349ab724c9182fdb237e2630b50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.378073 12389 master_service.cc:960] ReplaceTablet: received request to replace tablet ffac209a6cd8440289ee7765c3c24b40 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:41.378692 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_7d44b553bb6a405da5119c4cf46b4ce2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:41.393045 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d44b553bb6a405da5119c4cf46b4ce2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.403705 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 189eaca5fb2b4b0d83a2873e0495e5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.393890 12387 catalog_manager.cc:5838] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:41.422986 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 99465590a0e64b35b8a8f07e8777b524 (DEFAULT_TABLE table=default.Table_0c83b5f2aca049df9f95b55670a04502 [id=03889de8878549b7814b02a32caf20c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.424216 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99465590a0e64b35b8a8f07e8777b524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.438045 13203 ts_tablet_manager.cc:1950] T d38d785b63734ddd9f2e4f47ef55c7c9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:41.370366 13013 tablet_bootstrap.cc:492] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:41.447889 13012 ts_tablet_manager.cc:1434] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.172s user 0.005s sys 0.000s
I20260926 19:58:41.449374 12387 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet b82d75be43e841e7abcc9e9db800aa2d (table default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa]). Expected version 2 got 0
I20260926 19:58:41.452221 13032 ts_tablet_manager.cc:1929] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:41.452577 13032 log.cc:1199] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/d38d785b63734ddd9f2e4f47ef55c7c9
I20260926 19:58:41.453253 13032 ts_tablet_manager.cc:1950] T d38d785b63734ddd9f2e4f47ef55c7c9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:41.458525 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 99465590a0e64b35b8a8f07e8777b524 (DEFAULT_TABLE table=default.Table_0c83b5f2aca049df9f95b55670a04502 [id=03889de8878549b7814b02a32caf20c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.460285 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (DEFAULT_TABLE table=default.Table_5e21794a58a648c8a5806d4c535a8f37 [id=c2c079dc87ec480a82aa5b57b110da17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.471395 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (DEFAULT_TABLE table=default.Table_5e21794a58a648c8a5806d4c535a8f37 [id=c2c079dc87ec480a82aa5b57b110da17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.472587 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7d5e580f5645ab95fa03d5f669dc32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.480640 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_6cabacdbfcc94b7e876d1a62734db5a6" } modify_external_catalogs: true
I20260926 19:58:41.481163 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_6cabacdbfcc94b7e876d1a62734db5a6" } modify_external_catalogs: true
I20260926 19:58:41.485414 13048 raft_consensus.cc:515] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.485853 13239 tablet.cc:1751] T c1822d4c697d4c4985902355fc360dd1 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:41.488543 13092 tablet_bootstrap.cc:492] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:41.489035 13092 ts_tablet_manager.cc:1403] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.529s user 0.007s sys 0.005s
I20260926 19:58:41.491179 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (DEFAULT_TABLE table=default.Table_5e21794a58a648c8a5806d4c535a8f37 [id=c2c079dc87ec480a82aa5b57b110da17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.494553 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99465590a0e64b35b8a8f07e8777b524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.496567 13032 tablet_replica.cc:333] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:41.501753 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7d5e580f5645ab95fa03d5f669dc32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.502918 13012 tablet_bootstrap.cc:492] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:41.506429 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet d38d785b63734ddd9f2e4f47ef55c7c9 (table default.Table_199fcdc45b23492aacc9631f50874ddb [id=244a65d5edb64b3b8b8ca011dfbc9e24]) successfully deleted
I20260926 19:58:41.497287 13032 raft_consensus.cc:2243] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:41.507825 13032 raft_consensus.cc:2272] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.509665 13032 ts_tablet_manager.cc:1916] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:41.524683 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7d5e580f5645ab95fa03d5f669dc32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.524559 12389 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_14c589443e094e22bc1c798043fd6d04" } modify_external_catalogs: true
I20260926 19:58:41.537428 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet e550463602ff4b5f98e10dd593897bcf
I20260926 19:58:41.539161 12394 catalog_manager.cc:6934] ReplaceTablet: tablet e550463602ff4b5f98e10dd593897bcf deleted and replaced by tablet d48ee26c0aaa4abc8e08a1a2dadb028d
I20260926 19:58:41.541757 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet e550463602ff4b5f98e10dd593897bcf with delete_type TABLET_DATA_DELETED (replaced by tablet d48ee26c0aaa4abc8e08a1a2dadb028d) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:41.545559 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5aefddf9a623467fbcae8313cbfe329d" } new_table_name: "default.Table_21e4b470924b49518cf4ca4b8e336b36" modify_external_catalogs: true
I20260926 19:58:41.548280 13203 tablet_replica.cc:333] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:41.549041 13203 raft_consensus.cc:2243] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:41.549916 13032 ts_tablet_manager.cc:1929] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:41.550262 13032 log.cc:1199] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/8c683dc1272f4290a56f7fc7010ba038
I20260926 19:58:41.550899 13032 ts_tablet_manager.cc:1950] T 8c683dc1272f4290a56f7fc7010ba038 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:41.551585 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet e550463602ff4b5f98e10dd593897bcf with delete_type TABLET_DATA_DELETED (replaced by tablet d48ee26c0aaa4abc8e08a1a2dadb028d) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:41.525398 12389 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_14c589443e094e22bc1c798043fd6d04" } modify_external_catalogs: true
I20260926 19:58:41.555614 13203 pending_rounds.cc:70] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:41.555896 13203 pending_rounds.cc:77] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694344378257408 op_type: NO_OP noop_request { }
I20260926 19:58:41.555991 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 8c683dc1272f4290a56f7fc7010ba038 (table default.Table_c46bbc68304f4ebb80a240478a92349d [id=f1d58054ff284e03af21a1f7e857732e]) successfully deleted
I20260926 19:58:41.556252 13203 raft_consensus.cc:2889] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:41.556578 13203 raft_consensus.cc:2272] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.557605 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet e550463602ff4b5f98e10dd593897bcf with delete_type TABLET_DATA_DELETED (replaced by tablet d48ee26c0aaa4abc8e08a1a2dadb028d) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:41.557874 13032 tablet_replica.cc:333] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:41.558547 13032 raft_consensus.cc:2243] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260926 19:58:41.558563 12521 consensus_peers.cc:597] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:41.558957 13032 raft_consensus.cc:2272] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.558969 13203 ts_tablet_manager.cc:1916] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:41.560616 13032 ts_tablet_manager.cc:1916] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:41.561710 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet d38d785b63734ddd9f2e4f47ef55c7c9 (table default.Table_199fcdc45b23492aacc9631f50874ddb [id=244a65d5edb64b3b8b8ca011dfbc9e24]) successfully deleted
I20260926 19:58:41.561827 12760 raft_consensus.cc:1275] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Refusing update from remote peer f6e9d6439fd54b28925a68a6145d3dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:41.563519 13212 consensus_queue.cc:1048] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:41.566785 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 886ab0feae1e49629290bc892e518816 (DEFAULT_TABLE table=default.Table_9cf5aa5585a04712ab95e4b0d9428171 [id=6a6df456d54045ab95e8dc2c70856189]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.567595 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 886ab0feae1e49629290bc892e518816 (DEFAULT_TABLE table=default.Table_9cf5aa5585a04712ab95e4b0d9428171 [id=6a6df456d54045ab95e8dc2c70856189]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.567886 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 886ab0feae1e49629290bc892e518816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.568599 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 886ab0feae1e49629290bc892e518816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.578562 13240 raft_consensus.cc:493] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:41.580121 13013 tablet_bootstrap.cc:654] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:41.581501 13240 raft_consensus.cc:515] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.591918 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 886ab0feae1e49629290bc892e518816 (DEFAULT_TABLE table=default.Table_9cf5aa5585a04712ab95e4b0d9428171 [id=6a6df456d54045ab95e8dc2c70856189]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.592886 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 886ab0feae1e49629290bc892e518816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.596690 13032 ts_tablet_manager.cc:1929] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:41.597038 13032 log.cc:1199] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/1d25f740c1334c5cb53d7cffa3eb275e
I20260926 19:58:41.604286 13212 raft_consensus.cc:493] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:41.604740 13212 raft_consensus.cc:515] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.606246 13244 tablet.cc:1751] T c1822d4c697d4c4985902355fc360dd1 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:41.611022 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "381320b4d2844e2289a5ae390e6b0503" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:41.612104 12761 raft_consensus.cc:2468] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:41.645781 13032 ts_tablet_manager.cc:1950] T 1d25f740c1334c5cb53d7cffa3eb275e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:41.649003 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 1d25f740c1334c5cb53d7cffa3eb275e (table default.Table_b1794633093d4c15bfaa7f160828ff99 [id=c6f1da949bf242ae87b8bbdc6c1d334a]) successfully deleted
I20260926 19:58:41.650012 12924 raft_consensus.cc:3060] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.651504 13240 leader_election.cc:290] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:41.607842 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "381320b4d2844e2289a5ae390e6b0503" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:41.612280 13249 tablet.cc:1751] T c1822d4c697d4c4985902355fc360dd1 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:41.613904 13231 raft_consensus.cc:515] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.656476 12393 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_1305c705a9be4c02863b9f3d5d014850 [id=b1955ec80c98477c95b3b06cb73f3206]). Expected version 2 got 0
I20260926 19:58:41.657418 13231 leader_election.cc:290] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:41.619189 13048 leader_election.cc:290] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:41.627084 12857 leader_election.cc:343] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 99cf26cf4838454c98aa8337341ffd06
I20260926 19:58:41.660089 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46a5297ef854afd8ef783bf0692135e" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:41.660118 12392 catalog_manager.cc:5838] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:41.661777 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet c1822d4c697d4c4985902355fc360dd1 (table default.Table_1305c705a9be4c02863b9f3d5d014850 [id=b1955ec80c98477c95b3b06cb73f3206]). Expected version 2 got 0
W20260926 19:58:41.661900 12857 leader_election.cc:343] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: ef062e1775bd4a3c859124eaaba81399
I20260926 19:58:41.491660 13092 raft_consensus.cc:359] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.662822 13092 raft_consensus.cc:385] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:41.663091 13092 raft_consensus.cc:740] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:41.663836 13092 consensus_queue.cc:260] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.605968 13031 ts_tablet_manager.cc:1929] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:41.664661 13031 log.cc:1199] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/4eca1e5734324f99b9362915bdd00239
I20260926 19:58:41.665313 13031 ts_tablet_manager.cc:1950] T 4eca1e5734324f99b9362915bdd00239 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
W20260926 19:58:41.666983 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet b19f42511ad14d8a88fc0eaefb82e335 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b19f42511ad14d8a88fc0eaefb82e335 already in progress: creating tablet
I20260926 19:58:41.668587 13012 tablet_bootstrap.cc:654] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:41.668653 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet e550463602ff4b5f98e10dd593897bcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e550463602ff4b5f98e10dd593897bcf already in progress: creating tablet
I20260926 19:58:41.673102 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 56e337767be84507acdecfb189705bdd (DEFAULT_TABLE table=default.Table_6cabacdbfcc94b7e876d1a62734db5a6 [id=0d5fc67f634c45dab68955448173bfd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.674052 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e337767be84507acdecfb189705bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.646852 13246 tablet.cc:1751] T b46a5297ef854afd8ef783bf0692135e P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:41.608681 13212 leader_election.cc:290] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:41.614598 12519 leader_election.cc:304] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:41.638062 13247 tablet.cc:1751] T b46a5297ef854afd8ef783bf0692135e P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260926 19:58:41.682336 12521 leader_election.cc:343] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:41.686506 13212 raft_consensus.cc:2804] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:41.686869 13212 raft_consensus.cc:493] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:41.687601 13212 raft_consensus.cc:3060] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:41.689813 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 75316.8us
I20260926 19:58:41.694954 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99cf26cf4838454c98aa8337341ffd06" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:41.695462 12594 raft_consensus.cc:3060] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.698290 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b46a5297ef854afd8ef783bf0692135e" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:41.704140 13013 tablet_bootstrap.cc:492] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:41.704576 13013 ts_tablet_manager.cc:1403] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.334s user 0.009s sys 0.000s
I20260926 19:58:41.708621 13212 raft_consensus.cc:515] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.714789 13212 leader_election.cc:290] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:41.715986 12395 master_service.cc:960] ReplaceTablet: received request to replace tablet 3215bbd5be56438491a73925ac9ccbe6 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:41.716867 13012 tablet_bootstrap.cc:492] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:41.717281 13012 ts_tablet_manager.cc:1403] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.215s user 0.005s sys 0.004s
I20260926 19:58:41.718413 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "381320b4d2844e2289a5ae390e6b0503" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:41.728638 13012 raft_consensus.cc:359] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.729244 13012 raft_consensus.cc:385] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:41.729543 13012 raft_consensus.cc:740] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:41.732497 13012 consensus_queue.cc:260] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.736385 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_37d98bf446e04c2992e3325245a72122"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:41.737762 12760 raft_consensus.cc:1275] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Refusing update from remote peer f6e9d6439fd54b28925a68a6145d3dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:41.739235 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "381320b4d2844e2289a5ae390e6b0503" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
W20260926 19:58:41.739511 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37d98bf446e04c2992e3325245a72122 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.739777 12761 raft_consensus.cc:3060] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.744216 13048 raft_consensus.cc:493] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:41.744692 13048 raft_consensus.cc:515] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.749131 12761 raft_consensus.cc:2468] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:41.754810 13012 ts_tablet_manager.cc:1434] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.028s user 0.004s sys 0.000s
I20260926 19:58:41.756263 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_efa6ffdea9394a91a51887f7b9d3d37a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:41.758333 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_efa6ffdea9394a91a51887f7b9d3d37a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.759297 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9cf5aa5585a04712ab95e4b0d9428171" } new_table_name: "default.Table_4ceba47368094646b52f05a4bd441a64" modify_external_catalogs: true
I20260926 19:58:41.762475 13012 tablet_bootstrap.cc:492] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:41.764966 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 56e337767be84507acdecfb189705bdd
I20260926 19:58:41.766037 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 56e337767be84507acdecfb189705bdd (DEFAULT_TABLE table=default.Table_6cabacdbfcc94b7e876d1a62734db5a6 [id=0d5fc67f634c45dab68955448173bfd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.767655 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e337767be84507acdecfb189705bdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:41.769537 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 56e337767be84507acdecfb189705bdd because the tablet was not found. No further retry: Not found: Tablet not found: 56e337767be84507acdecfb189705bdd
I20260926 19:58:41.770673 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e337767be84507acdecfb189705bdd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:41.717211 12521 consensus_peers.cc:597] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:41.721243 13203 ts_tablet_manager.cc:1929] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:41.773826 13203 log.cc:1199] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/8c683dc1272f4290a56f7fc7010ba038
W20260926 19:58:41.773723 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 56640.1us
W20260926 19:58:41.774422 12521 leader_election.cc:343] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:41.774294 13128 consensus_queue.cc:1048] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:41.774631 13203 ts_tablet_manager.cc:1950] T 8c683dc1272f4290a56f7fc7010ba038 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:41.733552 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef062e1775bd4a3c859124eaaba81399" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:41.775627 12595 raft_consensus.cc:3060] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.783834 12519 leader_election.cc:304] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:41.784384 13203 tablet_replica.cc:333] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:41.785967 13128 raft_consensus.cc:2804] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:41.786350 13128 raft_consensus.cc:697] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:41.786921 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 8c683dc1272f4290a56f7fc7010ba038 (table default.Table_c46bbc68304f4ebb80a240478a92349d [id=f1d58054ff284e03af21a1f7e857732e]) successfully deleted
I20260926 19:58:41.787048 13128 consensus_queue.cc:237] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.790344 13246 tablet.cc:1751] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:41.712316 13013 raft_consensus.cc:359] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.762914 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 56e337767be84507acdecfb189705bdd (DEFAULT_TABLE table=default.Table_6cabacdbfcc94b7e876d1a62734db5a6 [id=0d5fc67f634c45dab68955448173bfd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.794255 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56e337767be84507acdecfb189705bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.768617 12564 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e337767be84507acdecfb189705bdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:41.797237 13247 tablet.cc:1751] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:41.801376 12595 raft_consensus.cc:2468] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:41.811514 13203 raft_consensus.cc:2243] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:41.812012 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 3dabe03472b640eca7d88e4c54cc2782 (DEFAULT_TABLE table=default.Table_f1a85473b0594a75947fbf9a21bd5469 [id=3f2e4633bd6c4004ab931d817248ed45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.813126 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dabe03472b640eca7d88e4c54cc2782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.813717 13203 raft_consensus.cc:2272] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.815644 13203 ts_tablet_manager.cc:1916] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:41.742717 12594 raft_consensus.cc:2468] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:41.810964 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 3dabe03472b640eca7d88e4c54cc2782 (DEFAULT_TABLE table=default.Table_f1a85473b0594a75947fbf9a21bd5469 [id=3f2e4633bd6c4004ab931d817248ed45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.822712 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dabe03472b640eca7d88e4c54cc2782. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:41.836061 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66c8916d88dd48998eab514bd3c538b9 has no consensus state
I20260926 19:58:41.840736 13012 tablet_bootstrap.cc:654] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:41.805634 13013 raft_consensus.cc:385] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:41.848186 13013 raft_consensus.cc:740] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:41.849941 13013 consensus_queue.cc:260] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.846172 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 60) took 1569 ms. Trace:
I20260926 19:58:41.853775 13031 rpcz_store.cc:276] 0926 19:58:40.276740 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:40.276870 (+ 130us) service_pool.cc:224] Handling call
0926 19:58:40.963382 (+686512us) ts_tablet_manager.cc:1151] Deleting tablet 4eca1e5734324f99b9362915bdd00239
0926 19:58:40.963423 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:41.416828 (+453405us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:41.605849 (+189021us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:41.846112 (+240263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":178268,"tablet-delete.queue_time_us":685909}
I20260926 19:58:41.859237 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_309601918a5743ab8547d309642f118e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:41.860813 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_309601918a5743ab8547d309642f118e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:41.858171 13092 ts_tablet_manager.cc:1434] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.369s user 0.006s sys 0.001s
I20260926 19:58:41.866648 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 67) took 1415 ms. Trace:
I20260926 19:58:41.868326 13031 rpcz_store.cc:276] 0926 19:58:40.450801 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:40.450940 (+ 139us) service_pool.cc:224] Handling call
0926 19:58:41.866324 (+1415384us) ts_tablet_manager.cc:1151] Deleting tablet 28b592db99c1495aafb555ff2a76a154
0926 19:58:41.866363 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:41.866596 (+ 233us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1413419}
I20260926 19:58:41.871647 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 68) took 1384 ms. Trace:
I20260926 19:58:41.873065 13031 rpcz_store.cc:276] 0926 19:58:40.487229 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:40.487373 (+ 144us) service_pool.cc:224] Handling call
0926 19:58:41.871349 (+1383976us) ts_tablet_manager.cc:1151] Deleting tablet ffac209a6cd8440289ee7765c3c24b40
0926 19:58:41.871385 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:41.871599 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1383413}
I20260926 19:58:41.875932 13260 tablet.cc:1751] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:41.867599 13092 tablet_bootstrap.cc:492] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:41.884057 13013 ts_tablet_manager.cc:1434] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.179s user 0.005s sys 0.005s
I20260926 19:58:41.884891 13013 tablet_bootstrap.cc:492] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:41.900461 13262 tablet.cc:1751] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:41.907254 12389 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet bdcecfe46c3447e493f1ab8b5eff3c62
I20260926 19:58:41.915974 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet bdcecfe46c3447e493f1ab8b5eff3c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:41.916492 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 55b7715b8fcb4b59917ae14a83f9955d (DEFAULT_TABLE table=default.Table_65a24dd219044e0f80e841c1254730d6 [id=d9c3ce1d8dd440c2862eb5fa4c58bf72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.917874 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b7715b8fcb4b59917ae14a83f9955d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.917384 13032 tablet_replica.cc:333] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:41.922192 13032 raft_consensus.cc:2243] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:41.922690 13032 raft_consensus.cc:2272] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.924688 13032 ts_tablet_manager.cc:1916] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:41.926241 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa82c030ec34ac5a8c448d80cdbae95 (DEFAULT_TABLE table=default.Table_a37fe86f467e42e59cecf9922dd1f820 [id=8016a9769e6d43d89397827367958311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.927330 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa82c030ec34ac5a8c448d80cdbae95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.946727 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa82c030ec34ac5a8c448d80cdbae95 (DEFAULT_TABLE table=default.Table_a37fe86f467e42e59cecf9922dd1f820 [id=8016a9769e6d43d89397827367958311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.947109 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 55b7715b8fcb4b59917ae14a83f9955d (DEFAULT_TABLE table=default.Table_65a24dd219044e0f80e841c1254730d6 [id=d9c3ce1d8dd440c2862eb5fa4c58bf72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:41.947733 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa82c030ec34ac5a8c448d80cdbae95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.948504 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b7715b8fcb4b59917ae14a83f9955d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:41.951939 12565 tablet_service.cc:1558] Processing DeleteTablet for tablet bdcecfe46c3447e493f1ab8b5eff3c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:41.953171 13203 ts_tablet_manager.cc:1929] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:41.954634 13013 tablet_bootstrap.cc:654] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:41.967777 12923 raft_consensus.cc:3060] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:41.970455 13203 log.cc:1199] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/ef062e1775bd4a3c859124eaaba81399
I20260926 19:58:41.971145 13203 ts_tablet_manager.cc:1950] T ef062e1775bd4a3c859124eaaba81399 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:41.983937 13031 tablet_replica.cc:333] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:41.984661 13031 raft_consensus.cc:2243] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:41.985140 13031 raft_consensus.cc:2272] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.986682 13203 tablet_replica.cc:333] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:41.987429 13203 raft_consensus.cc:2243] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:41.987874 13203 raft_consensus.cc:2272] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:41.990263 13012 tablet_bootstrap.cc:492] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:41.990653 13012 ts_tablet_manager.cc:1403] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.228s user 0.016s sys 0.004s
I20260926 19:58:41.993379 13012 raft_consensus.cc:359] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:41.997918 13012 raft_consensus.cc:385] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:41.998235 13012 raft_consensus.cc:740] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.002524 13032 ts_tablet_manager.cc:1929] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 19:58:42.002951 13032 log.cc:1199] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/bdcecfe46c3447e493f1ab8b5eff3c62
I20260926 19:58:42.003749 13032 ts_tablet_manager.cc:1950] T bdcecfe46c3447e493f1ab8b5eff3c62 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:42.007647 13012 consensus_queue.cc:260] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.012905 13012 ts_tablet_manager.cc:1434] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20260926 19:58:42.013955 13012 tablet_bootstrap.cc:492] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:42.022159 13203 ts_tablet_manager.cc:1916] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:42.023098 12381 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.254:33143, user_credentials={real_user=slave}} blocked reactor thread for 103271us
I20260926 19:58:42.024206 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet ef062e1775bd4a3c859124eaaba81399 (table default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]) successfully deleted
W20260926 19:58:42.024971 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 6f1dafade40a46649ac2780b8ad3b9ff because the tablet was not found. No further retry: Not found: Tablet not found: 6f1dafade40a46649ac2780b8ad3b9ff
I20260926 19:58:42.032423 13013 tablet_bootstrap.cc:492] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:42.032909 13013 ts_tablet_manager.cc:1403] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.148s user 0.009s sys 0.005s
I20260926 19:58:42.034669 13012 tablet_bootstrap.cc:654] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.039242 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_14c589443e094e22bc1c798043fd6d04 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]) successfully deleted
I20260926 19:58:42.038843 13013 raft_consensus.cc:359] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.058212 13013 raft_consensus.cc:385] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.058503 13013 raft_consensus.cc:740] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.063020 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:42.068233 13013 consensus_queue.cc:260] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.071171 13048 leader_election.cc:290] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:42.072783 13012 tablet_bootstrap.cc:492] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:42.072675 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9acbd4e5aa23433d860d399d78485d0d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:42.073195 13012 ts_tablet_manager.cc:1403] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.002s
I20260926 19:58:42.076833 13013 ts_tablet_manager.cc:1434] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.044s user 0.003s sys 0.004s
I20260926 19:58:42.077726 13013 tablet_bootstrap.cc:492] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:42.076946 13012 raft_consensus.cc:359] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:42.078593 13012 raft_consensus.cc:385] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.078848 13012 raft_consensus.cc:740] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.081981 12395 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 3215bbd5be56438491a73925ac9ccbe6
I20260926 19:58:42.083241 12395 catalog_manager.cc:6934] ReplaceTablet: tablet 3215bbd5be56438491a73925ac9ccbe6 deleted and replaced by tablet 1d0583424ef64d9ea3beeb1c01ceafba
I20260926 19:58:42.084240 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d0583424ef64d9ea3beeb1c01ceafba) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:42.085767 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 3215bbd5be56438491a73925ac9ccbe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3215bbd5be56438491a73925ac9ccbe6 already in progress: creating tablet
I20260926 19:58:42.085884 13012 consensus_queue.cc:260] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:42.088550 13012 ts_tablet_manager.cc:1434] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.015s user 0.006s sys 0.002s
I20260926 19:58:42.093485 12394 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_1281608b01e84b90b871e71e1adc84b9 [id=a774283438c64c668a5b2e9e4d62ed99] alter complete (version 1)
I20260926 19:58:42.093593 13012 tablet_bootstrap.cc:492] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:42.103565 13013 tablet_bootstrap.cc:654] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.103966 13149 raft_consensus.cc:493] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.104382 13149 raft_consensus.cc:515] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.106281 13149 leader_election.cc:290] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:58:42.108397 12695 leader_election.cc:343] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 28b592db99c1495aafb555ff2a76a154
I20260926 19:58:42.107857 13012 tablet_bootstrap.cc:654] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.112099 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d0583424ef64d9ea3beeb1c01ceafba) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:42.116307 13203 ts_tablet_manager.cc:1929] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260926 19:58:42.116827 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 886ab0feae1e49629290bc892e518816 consensus state has no leader
I20260926 19:58:42.121073 13092 tablet_bootstrap.cc:654] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.124655 13203 log.cc:1199] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/99cf26cf4838454c98aa8337341ffd06
I20260926 19:58:42.126921 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28b592db99c1495aafb555ff2a76a154" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:42.127933 13203 ts_tablet_manager.cc:1950] T 99cf26cf4838454c98aa8337341ffd06 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
W20260926 19:58:42.128846 12696 leader_election.cc:343] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.129324 12696 leader_election.cc:304] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:42.130169 13149 raft_consensus.cc:2749] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.132275 13012 tablet_bootstrap.cc:492] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:42.132712 13012 ts_tablet_manager.cc:1403] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.039s user 0.009s sys 0.000s
I20260926 19:58:42.134646 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 79) took 1161 ms. Trace:
I20260926 19:58:42.136610 13012 raft_consensus.cc:359] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.137290 13012 raft_consensus.cc:385] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.139421 13012 raft_consensus.cc:740] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.140242 13012 consensus_queue.cc:260] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.142206 13012 ts_tablet_manager.cc:1434] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260926 19:58:42.142987 13012 tablet_bootstrap.cc:492] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:41.872648 12858 leader_election.cc:304] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:58:42.148507 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 404543us
I20260926 19:58:42.151219 13013 tablet_bootstrap.cc:492] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:42.159904 13013 ts_tablet_manager.cc:1403] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.082s user 0.009s sys 0.002s
I20260926 19:58:42.162667 13013 raft_consensus.cc:359] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:42.170018 13013 raft_consensus.cc:385] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.149134 13203 rpcz_store.cc:276] 0926 19:58:40.972797 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:40.979964 (+ 7167us) service_pool.cc:224] Handling call
0926 19:58:41.973136 (+993172us) ts_tablet_manager.cc:1151] Deleting tablet 99cf26cf4838454c98aa8337341ffd06
0926 19:58:41.986555 (+ 13419us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:42.065065 (+ 78510us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:42.116182 (+ 51117us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:42.134580 (+ 18398us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":992675}
I20260926 19:58:42.170838 13012 tablet_bootstrap.cc:654] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.171442 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 99cf26cf4838454c98aa8337341ffd06 (table default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]) successfully deleted
W20260926 19:58:42.172936 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet b19f42511ad14d8a88fc0eaefb82e335 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b19f42511ad14d8a88fc0eaefb82e335 already in progress: creating tablet
W20260926 19:58:42.174638 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet e550463602ff4b5f98e10dd593897bcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e550463602ff4b5f98e10dd593897bcf already in progress: creating tablet
W20260926 19:58:42.176080 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 56e337767be84507acdecfb189705bdd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56e337767be84507acdecfb189705bdd already in progress: creating tablet
I20260926 19:58:42.177018 13203 tablet_replica.cc:333] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:42.177769 13203 raft_consensus.cc:2243] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:42.179319 13203 raft_consensus.cc:2272] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.180439 13249 tablet.cc:1751] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:42.187583 13203 ts_tablet_manager.cc:1916] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.178100 13013 raft_consensus.cc:740] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.190701 13013 consensus_queue.cc:260] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:42.192623 13012 tablet_bootstrap.cc:492] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:42.193011 13012 ts_tablet_manager.cc:1403] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.050s user 0.007s sys 0.002s
I20260926 19:58:42.195425 13012 raft_consensus.cc:359] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.196008 13012 raft_consensus.cc:385] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.196261 13012 raft_consensus.cc:740] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.197781 13013 ts_tablet_manager.cc:1434] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20260926 19:58:42.202482 13013 tablet_bootstrap.cc:492] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:42.197005 13012 consensus_queue.cc:260] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.209679 13012 ts_tablet_manager.cc:1434] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.016s user 0.003s sys 0.003s
I20260926 19:58:42.210624 13012 tablet_bootstrap.cc:492] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:42.211854 12380 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 886ab0feae1e49629290bc892e518816 (table default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 886ab0feae1e49629290bc892e518816 consensus state has no leader
I20260926 19:58:42.125398 12394 catalog_manager.cc:5838] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:42.216373 12394 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:42.218008 13128 raft_consensus.cc:493] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.218535 13128 raft_consensus.cc:515] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.219856 13012 tablet_bootstrap.cc:654] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.220798 13128 leader_election.cc:290] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:42.228690 13013 tablet_bootstrap.cc:654] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.232755 13031 ts_tablet_manager.cc:1916] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:42.232944 12519 leader_election.cc:343] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 1d25f740c1334c5cb53d7cffa3eb275e
I20260926 19:58:42.240458 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9acbd4e5aa23433d860d399d78485d0d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:42.243196 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 3dabe03472b640eca7d88e4c54cc2782 (DEFAULT_TABLE table=default.Table_f1a85473b0594a75947fbf9a21bd5469 [id=3f2e4633bd6c4004ab931d817248ed45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.243871 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa82c030ec34ac5a8c448d80cdbae95 (DEFAULT_TABLE table=default.Table_a37fe86f467e42e59cecf9922dd1f820 [id=8016a9769e6d43d89397827367958311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.244179 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dabe03472b640eca7d88e4c54cc2782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.244810 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa82c030ec34ac5a8c448d80cdbae95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.243196 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d25f740c1334c5cb53d7cffa3eb275e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:42.253086 12521 leader_election.cc:343] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.253584 12521 leader_election.cc:304] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:42.254549 13128 raft_consensus.cc:2749] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.256412 13149 raft_consensus.cc:493] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.256867 13149 raft_consensus.cc:515] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.258790 13149 leader_election.cc:290] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:42.260907 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732a34f4894940b3889bab5f0c0891e0" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:42.261343 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732a34f4894940b3889bab5f0c0891e0" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
W20260926 19:58:42.262182 12696 leader_election.cc:343] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:42.262913 12695 leader_election.cc:343] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.263286 12695 leader_election.cc:304] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:42.263947 13012 tablet_bootstrap.cc:492] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:42.264317 13012 ts_tablet_manager.cc:1403] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.054s user 0.005s sys 0.006s
I20260926 19:58:42.264467 13180 raft_consensus.cc:2749] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.265434 13180 consensus_queue.cc:1048] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:42.267103 13012 raft_consensus.cc:359] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.267697 13012 raft_consensus.cc:385] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.267935 13012 raft_consensus.cc:740] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.272135 13012 consensus_queue.cc:260] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.274179 13012 ts_tablet_manager.cc:1434] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20260926 19:58:42.274971 13012 tablet_bootstrap.cc:492] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:42.279580 13013 tablet_bootstrap.cc:492] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:42.279642 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e337767be84507acdecfb189705bdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:42.280030 13013 ts_tablet_manager.cc:1403] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.078s user 0.009s sys 0.002s
I20260926 19:58:42.282027 12858 leader_election.cc:304] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:42.284050 13128 raft_consensus.cc:493] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.284528 13128 raft_consensus.cc:515] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:42.287829 13128 leader_election.cc:290] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:42.287812 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8875d46f7c46819ee67e650b098c27" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:42.288410 12760 raft_consensus.cc:2468] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:42.282387 13013 raft_consensus.cc:359] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.289705 12519 leader_election.cc:304] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:42.290122 13013 raft_consensus.cc:385] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.290378 13013 raft_consensus.cc:740] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.290691 13128 raft_consensus.cc:2804] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:42.291044 13128 raft_consensus.cc:493] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:42.291417 13128 raft_consensus.cc:3060] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:42.291165 13013 consensus_queue.cc:260] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.284276 13203 ts_tablet_manager.cc:1929] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 19:58:42.292124 13203 log.cc:1199] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/bdcecfe46c3447e493f1ab8b5eff3c62
I20260926 19:58:42.293777 13203 ts_tablet_manager.cc:1950] T bdcecfe46c3447e493f1ab8b5eff3c62 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:42.295919 13013 ts_tablet_manager.cc:1434] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260926 19:58:42.299841 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_14c589443e094e22bc1c798043fd6d04 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]) successfully deleted
I20260926 19:58:42.302953 13012 tablet_bootstrap.cc:654] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.303238 13013 tablet_bootstrap.cc:492] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:42.304441 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_27371e4eb0cd4fa48aaef20d2db04fcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:42.305891 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27371e4eb0cd4fa48aaef20d2db04fcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:42.307559 13128 raft_consensus.cc:515] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:42.316041 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 3215bbd5be56438491a73925ac9ccbe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3215bbd5be56438491a73925ac9ccbe6 already in progress: creating tablet
I20260926 19:58:42.320626 12388 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_29ce86fb5af04ec4b8411a4f3135b79d" } modify_external_catalogs: true
I20260926 19:58:42.321123 12388 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_29ce86fb5af04ec4b8411a4f3135b79d" } modify_external_catalogs: true
I20260926 19:58:42.322896 12396 master_service.cc:960] ReplaceTablet: received request to replace tablet 1d25f740c1334c5cb53d7cffa3eb275e from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:42.324045 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_72fe02818135414ca8fb8033ebda39c7" } new_table_name: "default.Table_d739366722334ab9a135d008dccea789" modify_external_catalogs: true
I20260926 19:58:42.327430 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8875d46f7c46819ee67e650b098c27" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:42.327963 12760 raft_consensus.cc:3060] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:42.331128 12393 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_1305c705a9be4c02863b9f3d5d014850 [id=b1955ec80c98477c95b3b06cb73f3206] alter complete (version 2)
I20260926 19:58:42.334937 12760 raft_consensus.cc:2468] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:42.338973 13012 tablet_bootstrap.cc:492] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:42.339439 13012 ts_tablet_manager.cc:1403] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.005s
I20260926 19:58:42.283115 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 55b7715b8fcb4b59917ae14a83f9955d (DEFAULT_TABLE table=default.Table_65a24dd219044e0f80e841c1254730d6 [id=d9c3ce1d8dd440c2862eb5fa4c58bf72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.289748 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8875d46f7c46819ee67e650b098c27" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:42.341714 13012 raft_consensus.cc:359] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.342303 13012 raft_consensus.cc:385] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.342576 13012 raft_consensus.cc:740] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.343118 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b7715b8fcb4b59917ae14a83f9955d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.308880 13013 tablet_bootstrap.cc:654] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.336027 12519 leader_election.cc:304] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
W20260926 19:58:42.344224 12521 leader_election.cc:343] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.344440 13261 raft_consensus.cc:2804] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:42.345353 13261 raft_consensus.cc:697] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:42.292225 13031 ts_tablet_manager.cc:1929] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:42.346017 13031 log.cc:1199] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/d38d785b63734ddd9f2e4f47ef55c7c9
I20260926 19:58:42.346123 13261 consensus_queue.cc:237] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:42.346812 13031 ts_tablet_manager.cc:1950] T d38d785b63734ddd9f2e4f47ef55c7c9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:42.348331 13149 raft_consensus.cc:493] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.348958 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 69) took 1394 ms. Trace:
I20260926 19:58:42.348748 13149 raft_consensus.cc:515] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.300448 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet bdcecfe46c3447e493f1ab8b5eff3c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:42.350540 13149 leader_election.cc:290] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:42.351297 13270 tablet.cc:1751] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:42.351943 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6d07cb4c954f23be80b315a5748adc" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:42.352870 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6d07cb4c954f23be80b315a5748adc" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
W20260926 19:58:42.353257 12696 leader_election.cc:343] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.353492 12595 raft_consensus.cc:2468] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:42.355489 12695 leader_election.cc:304] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:42.357432 13149 raft_consensus.cc:2804] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:42.357796 13149 raft_consensus.cc:493] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:42.358099 13149 raft_consensus.cc:3060] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:42.358371 13048 raft_consensus.cc:2749] T b46a5297ef854afd8ef783bf0692135e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.363189 13128 leader_election.cc:290] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:42.364950 13092 tablet_bootstrap.cc:492] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:42.365374 13092 ts_tablet_manager.cc:1403] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.498s user 0.006s sys 0.006s
I20260926 19:58:42.365815 13048 raft_consensus.cc:2804] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:42.366480 13048 raft_consensus.cc:697] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:42.367223 13048 consensus_queue.cc:237] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.370355 13149 raft_consensus.cc:515] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:42.372445 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 90510.7us
I20260926 19:58:42.375937 12387 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c7b5800561444fd7ad044784765fd2bb" } modify_external_catalogs: true
I20260926 19:58:42.376632 12387 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c7b5800561444fd7ad044784765fd2bb" } modify_external_catalogs: true
I20260926 19:58:42.382236 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8875d46f7c46819ee67e650b098c27" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:42.383555 12521 leader_election.cc:343] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.373138 12858 leader_election.cc:304] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:42.384119 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6d07cb4c954f23be80b315a5748adc" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:42.384687 13231 raft_consensus.cc:2804] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
W20260926 19:58:42.385635 12696 leader_election.cc:343] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.375231 13012 consensus_queue.cc:260] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.386193 13180 raft_consensus.cc:493] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.386571 13180 raft_consensus.cc:515] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.390362 13180 leader_election.cc:290] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:42.392246 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b19f42511ad14d8a88fc0eaefb82e335" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:42.394495 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b19f42511ad14d8a88fc0eaefb82e335" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:42.396516 12696 leader_election.cc:343] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.397338 13128 raft_consensus.cc:493] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.397930 13128 raft_consensus.cc:515] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:42.399277 12395 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2411a153e7194406a2bb7b879c08d7dd consensus state has no leader
I20260926 19:58:42.400326 13128 leader_election.cc:290] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:42.399143 12858 leader_election.cc:304] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:42.403599 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffac209a6cd8440289ee7765c3c24b40" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:42.405114 12521 leader_election.cc:343] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.406772 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 4e82262ea1c441808e9765aef911c903
W20260926 19:58:42.401647 12695 leader_election.cc:343] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.408056 12695 leader_election.cc:304] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:42.409667 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e82262ea1c441808e9765aef911c903 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:42.409729 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e82262ea1c441808e9765aef911c903 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:42.410344 13032 tablet_replica.cc:333] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:42.411015 13032 raft_consensus.cc:2243] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 19:58:42.411134 12519 leader_election.cc:343] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: ffac209a6cd8440289ee7765c3c24b40
I20260926 19:58:42.411492 13032 raft_consensus.cc:2272] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.411538 12519 leader_election.cc:304] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:42.413787 13203 tablet_replica.cc:333] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:42.414515 13203 raft_consensus.cc:2243] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:42.413341 13032 ts_tablet_manager.cc:1916] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.415611 13180 raft_consensus.cc:2749] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.425710 13012 ts_tablet_manager.cc:1434] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.086s user 0.006s sys 0.000s
I20260926 19:58:42.428897 12391 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 9acbd4e5aa23433d860d399d78485d0d (table default.Table_e0ad88ad494c42759968cb1dc8501282 [id=6890560cf0104034a9eb9431a38c99ec]). Expected version 1 got 0
I20260926 19:58:42.429548 12392 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_e0ad88ad494c42759968cb1dc8501282 [id=6890560cf0104034a9eb9431a38c99ec] alter complete (version 1)
I20260926 19:58:42.442327 13149 leader_election.cc:290] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:42.443727 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6d07cb4c954f23be80b315a5748adc" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:42.444265 12595 raft_consensus.cc:3060] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:42.446398 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 47330.5us
I20260926 19:58:42.451409 12392 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa] alter complete (version 2)
I20260926 19:58:42.453197 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 28b592db99c1495aafb555ff2a76a154 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a7d5e580f5645ab95fa03d5f669dc32) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:42.349377 13031 rpcz_store.cc:276] 0926 19:58:40.954837 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:40.954958 (+ 121us) service_pool.cc:224] Handling call
0926 19:58:41.983767 (+1028809us) ts_tablet_manager.cc:1151] Deleting tablet d38d785b63734ddd9f2e4f47ef55c7c9
0926 19:58:41.983825 (+ 58us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:42.272487 (+288662us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:42.292116 (+ 19629us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:42.348901 (+ 56785us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":925249}
I20260926 19:58:42.458109 13031 tablet_replica.cc:333] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:42.458793 13031 raft_consensus.cc:2243] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:42.459213 13031 raft_consensus.cc:2272] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.459365 12595 raft_consensus.cc:2468] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:42.460709 12695 leader_election.cc:304] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:42.461206 13283 raft_consensus.cc:2749] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:42.462021 12382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.253:37913, user_credentials={real_user=slave}} blocked reactor thread for 86331.1us
I20260926 19:58:42.466192 13149 raft_consensus.cc:2804] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:42.466635 13149 raft_consensus.cc:697] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:42.467350 13149 consensus_queue.cc:237] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:42.468317 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 28b592db99c1495aafb555ff2a76a154 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28b592db99c1495aafb555ff2a76a154 already in progress: creating tablet
W20260926 19:58:42.468899 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet ffac209a6cd8440289ee7765c3c24b40 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffac209a6cd8440289ee7765c3c24b40 already in progress: creating tablet
I20260926 19:58:42.479712 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:42.482555 13261 raft_consensus.cc:493] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:42.482692 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 71588us
I20260926 19:58:42.483002 13261 raft_consensus.cc:515] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.483387 13203 raft_consensus.cc:2272] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.485210 13203 ts_tablet_manager.cc:1916] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.488481 13012 tablet_bootstrap.cc:492] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:42.461611 13231 raft_consensus.cc:697] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:42.470793 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d0583424ef64d9ea3beeb1c01ceafba) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:42.493607 13231 consensus_queue.cc:237] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.495411 13128 raft_consensus.cc:2749] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.473548 13092 raft_consensus.cc:359] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.489035 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3f3931a71048fd874c1131d44f7b00" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:42.496405 13261 leader_election.cc:290] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:42.496974 13092 raft_consensus.cc:385] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.497538 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3f3931a71048fd874c1131d44f7b00" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:42.497856 13092 raft_consensus.cc:740] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.498104 12760 raft_consensus.cc:2468] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:42.498571 13092 consensus_queue.cc:260] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.499382 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet d38d785b63734ddd9f2e4f47ef55c7c9 (table default.Table_199fcdc45b23492aacc9631f50874ddb [id=244a65d5edb64b3b8b8ca011dfbc9e24]) successfully deleted
I20260926 19:58:42.499565 12519 leader_election.cc:304] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:42.500731 13092 ts_tablet_manager.cc:1434] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.135s user 0.005s sys 0.000s
I20260926 19:58:42.500906 13128 raft_consensus.cc:2804] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:42.501330 13128 raft_consensus.cc:493] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:42.501662 13031 ts_tablet_manager.cc:1916] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.501610 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e82262ea1c441808e9765aef911c903 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:42.501773 13128 raft_consensus.cc:3060] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:42.505769 12521 leader_election.cc:343] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.507786 13013 tablet_bootstrap.cc:492] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
W20260926 19:58:42.519546 12696 consensus_peers.cc:597] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:42.520503 13013 ts_tablet_manager.cc:1403] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.217s user 0.013s sys 0.016s
I20260926 19:58:42.515322 12391 catalog_manager.cc:5838] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:42.527630 13092 tablet_bootstrap.cc:492] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:42.532891 13092 tablet_bootstrap.cc:654] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.522622 13013 raft_consensus.cc:359] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.539662 13013 raft_consensus.cc:385] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.539920 13013 raft_consensus.cc:740] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.540355 13032 ts_tablet_manager.cc:1929] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:42.540706 13032 log.cc:1199] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/4e82262ea1c441808e9765aef911c903
I20260926 19:58:42.540514 13013 consensus_queue.cc:260] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.541386 13032 ts_tablet_manager.cc:1950] T 4e82262ea1c441808e9765aef911c903 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:42.542371 13013 ts_tablet_manager.cc:1434] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260926 19:58:42.546428 13203 ts_tablet_manager.cc:1929] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:42.546775 13203 log.cc:1199] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/4e82262ea1c441808e9765aef911c903
I20260926 19:58:42.549937 13013 tablet_bootstrap.cc:492] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:42.556037 13013 tablet_bootstrap.cc:654] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.562680 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet ffac209a6cd8440289ee7765c3c24b40 with delete_type TABLET_DATA_DELETED (replaced by tablet 189eaca5fb2b4b0d83a2873e0495e5ff) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:42.566423 13130 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
I20260926 19:58:42.570714 13128 raft_consensus.cc:515] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.572829 13203 ts_tablet_manager.cc:1950] T 4e82262ea1c441808e9765aef911c903 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
W20260926 19:58:42.573536 13130 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:58:42.574331 13130 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260926 19:58:42.575181 13012 tablet_bootstrap.cc:654] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.575565 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 4e82262ea1c441808e9765aef911c903 (table default.Table_c7b5800561444fd7ad044784765fd2bb [id=68ea017f68a5467c8c1d1c1ea98f4701]) successfully deleted
I20260926 19:58:42.577425 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3f3931a71048fd874c1131d44f7b00" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:42.579120 12521 leader_election.cc:343] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.580001 13031 ts_tablet_manager.cc:1929] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:42.580355 13031 log.cc:1199] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/8c683dc1272f4290a56f7fc7010ba038
I20260926 19:58:42.580950 13031 ts_tablet_manager.cc:1950] T 8c683dc1272f4290a56f7fc7010ba038 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:42.581107 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3f3931a71048fd874c1131d44f7b00" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:42.581786 12760 raft_consensus.cc:3060] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:42.583413 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 4e82262ea1c441808e9765aef911c903 (table default.Table_c7b5800561444fd7ad044784765fd2bb [id=68ea017f68a5467c8c1d1c1ea98f4701]) successfully deleted
I20260926 19:58:42.585034 13032 tablet_replica.cc:333] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:42.585760 13032 raft_consensus.cc:2243] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:42.586182 13032 raft_consensus.cc:2272] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.587788 13032 ts_tablet_manager.cc:1916] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.588346 12760 raft_consensus.cc:2468] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:42.589457 12519 leader_election.cc:304] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:42.590329 13261 raft_consensus.cc:2804] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:42.590781 13261 raft_consensus.cc:697] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:42.591511 13261 consensus_queue.cc:237] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.593148 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 70) took 1607 ms. Trace:
I20260926 19:58:42.606544 13031 rpcz_store.cc:276] 0926 19:58:40.985432 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:40.986593 (+ 1161us) service_pool.cc:224] Handling call
0926 19:58:42.457961 (+1471368us) ts_tablet_manager.cc:1151] Deleting tablet 8c683dc1272f4290a56f7fc7010ba038
0926 19:58:42.458004 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:42.560249 (+102245us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:42.579886 (+ 19637us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:42.593092 (+ 13206us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1469262}
I20260926 19:58:42.607798 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 8c683dc1272f4290a56f7fc7010ba038 (table default.Table_c46bbc68304f4ebb80a240478a92349d [id=f1d58054ff284e03af21a1f7e857732e]) successfully deleted
I20260926 19:58:42.607929 13212 consensus_queue.cc:1048] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:42.610743 13031 tablet_replica.cc:333] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:42.611469 13031 raft_consensus.cc:2243] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:42.612076 13031 pending_rounds.cc:70] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Trying to abort 1 pending ops.
I20260926 19:58:42.612293 13031 pending_rounds.cc:77] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694350825828352 op_type: NO_OP noop_request { }
I20260926 19:58:42.612624 13031 raft_consensus.cc:2889] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:42.617816 13092 tablet_bootstrap.cc:492] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:42.618227 13092 ts_tablet_manager.cc:1403] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.091s user 0.003s sys 0.011s
I20260926 19:58:42.620714 13092 raft_consensus.cc:359] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.623205 13013 tablet_bootstrap.cc:492] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:42.623608 13013 ts_tablet_manager.cc:1403] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.081s user 0.008s sys 0.003s
I20260926 19:58:42.621392 13092 raft_consensus.cc:385] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.627058 13031 raft_consensus.cc:2272] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.626899 13092 raft_consensus.cc:740] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.627930 13092 consensus_queue.cc:260] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.630524 13031 ts_tablet_manager.cc:1916] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.635462 13128 leader_election.cc:290] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:42.636615 13013 raft_consensus.cc:359] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.638967 13012 tablet_bootstrap.cc:492] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:42.639453 13012 ts_tablet_manager.cc:1403] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.207s user 0.008s sys 0.006s
I20260926 19:58:42.639729 13203 tablet_replica.cc:333] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:42.640472 13203 raft_consensus.cc:2243] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:42.641585 13013 raft_consensus.cc:385] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.641731 13203 raft_consensus.cc:2272] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.641878 13013 raft_consensus.cc:740] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.642602 13013 consensus_queue.cc:260] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.643435 13203 ts_tablet_manager.cc:1916] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.644402 13013 ts_tablet_manager.cc:1434] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.021s user 0.003s sys 0.003s
I20260926 19:58:42.645859 13012 raft_consensus.cc:359] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.646445 13012 raft_consensus.cc:385] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.647307 13092 ts_tablet_manager.cc:1434] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260926 19:58:42.649276 13092 tablet_bootstrap.cc:492] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:42.666465 13013 tablet_bootstrap.cc:492] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:42.646679 13012 raft_consensus.cc:740] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.670122 13012 consensus_queue.cc:260] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.679150 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_cc636537cf0c4914bb56b0c214adbf55" } new_table_name: "default.Table_cb7930cd5c56467d9d6ccf644263f999" modify_external_catalogs: true
I20260926 19:58:42.680025 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_1b16c5189a4649b181b45752c6b81e4a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:42.680389 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a195c0af637546dc947815ce7a384b8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:42.681759 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b16c5189a4649b181b45752c6b81e4a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:42.681823 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a195c0af637546dc947815ce7a384b8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:42.682847 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b79b586799c14e0ab126d0dc3d462136"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:42.684167 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b79b586799c14e0ab126d0dc3d462136 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:42.691054 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_06d7f301c3cb4d0e8d374df103fab2f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:42.692708 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06d7f301c3cb4d0e8d374df103fab2f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:42.699291 13032 ts_tablet_manager.cc:1929] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:42.702487 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_0c83b5f2aca049df9f95b55670a04502" } new_table_name: "default.Table_3f85639b2411482d9062c58f0a8bdbb3" modify_external_catalogs: true
I20260926 19:58:42.705646 13203 ts_tablet_manager.cc:1929] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:42.705987 13203 log.cc:1199] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/1d25f740c1334c5cb53d7cffa3eb275e
I20260926 19:58:42.706606 13203 ts_tablet_manager.cc:1950] T 1d25f740c1334c5cb53d7cffa3eb275e P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:42.711099 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 1d25f740c1334c5cb53d7cffa3eb275e (table default.Table_b1794633093d4c15bfaa7f160828ff99 [id=c6f1da949bf242ae87b8bbdc6c1d334a]) successfully deleted
I20260926 19:58:42.714123 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1" } modify_external_catalogs: true
I20260926 19:58:42.714617 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1" } modify_external_catalogs: true
I20260926 19:58:42.720304 13032 log.cc:1199] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/28b592db99c1495aafb555ff2a76a154
I20260926 19:58:42.721006 13032 ts_tablet_manager.cc:1950] T 28b592db99c1495aafb555ff2a76a154 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:42.724575 12387 master_service.cc:960] ReplaceTablet: received request to replace tablet 886ab0feae1e49629290bc892e518816 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:42.729468 13013 tablet_bootstrap.cc:654] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:42.734719 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ff071a16f1244cebc8c93e5508280a3 has no consensus state
W20260926 19:58:42.738929 12521 consensus_peers.cc:597] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:42.739044 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 28b592db99c1495aafb555ff2a76a154 (table default.Table_5e21794a58a648c8a5806d4c535a8f37 [id=c2c079dc87ec480a82aa5b57b110da17]) successfully deleted
I20260926 19:58:42.740844 13203 tablet_replica.cc:333] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:42.746778 13012 ts_tablet_manager.cc:1434] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.107s user 0.001s sys 0.003s
I20260926 19:58:42.746901 13203 raft_consensus.cc:2243] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:42.747376 13203 raft_consensus.cc:2272] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.752995 13012 tablet_bootstrap.cc:492] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:42.766794 13092 tablet_bootstrap.cc:700] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent opening tablet: real 0.113s user 0.004s sys 0.000s
I20260926 19:58:42.767808 13092 tablet_bootstrap.cc:654] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.769099 13203 ts_tablet_manager.cc:1916] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.783179 13149 raft_consensus.cc:493] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.783648 13149 raft_consensus.cc:515] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.784304 12393 master_service.cc:960] ReplaceTablet: received request to replace tablet 0fd184e6295d47aeba809be6aab9ce1c from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:42.785543 13149 leader_election.cc:290] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:42.787408 13128 raft_consensus.cc:493] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.787832 13128 raft_consensus.cc:515] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.789728 13128 leader_election.cc:290] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:42.790875 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea151df659944f9a094fe9410c3f4bb" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:42.792472 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea151df659944f9a094fe9410c3f4bb" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:42.795651 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed45552637024bec99a76da079f352ec" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:42.797286 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99465590a0e64b35b8a8f07e8777b524 consensus state has no leader
W20260926 19:58:42.797197 12695 leader_election.cc:343] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:42.798413 12696 leader_election.cc:343] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.798768 12696 leader_election.cc:304] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:58:42.799834 12519 leader_election.cc:343] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.800410 13203 ts_tablet_manager.cc:1929] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:42.800738 13203 log.cc:1199] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/ffac209a6cd8440289ee7765c3c24b40
I20260926 19:58:42.800592 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed45552637024bec99a76da079f352ec" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:42.801915 12521 leader_election.cc:343] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:42.802268 12521 leader_election.cc:304] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:42.803085 13128 raft_consensus.cc:2749] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.803754 13149 raft_consensus.cc:2749] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.805965 13092 tablet_bootstrap.cc:492] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:42.806421 13092 ts_tablet_manager.cc:1403] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.157s user 0.009s sys 0.007s
I20260926 19:58:42.801355 13203 ts_tablet_manager.cc:1950] T ffac209a6cd8440289ee7765c3c24b40 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:42.809650 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet ffac209a6cd8440289ee7765c3c24b40 (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]) successfully deleted
I20260926 19:58:42.809329 13092 raft_consensus.cc:359] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.810876 13092 raft_consensus.cc:385] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.811121 13092 raft_consensus.cc:740] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.811795 13092 consensus_queue.cc:260] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.813898 13092 ts_tablet_manager.cc:1434] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260926 19:58:42.818485 13013 tablet_bootstrap.cc:492] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:42.818871 13013 ts_tablet_manager.cc:1403] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.153s user 0.005s sys 0.005s
I20260926 19:58:42.822227 13013 raft_consensus.cc:359] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.822803 13013 raft_consensus.cc:385] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.823019 13013 raft_consensus.cc:740] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.823666 13013 consensus_queue.cc:260] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.825244 13092 tablet_bootstrap.cc:492] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:42.828053 13031 ts_tablet_manager.cc:1929] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:42.828496 13031 log.cc:1199] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/ef062e1775bd4a3c859124eaaba81399
I20260926 19:58:42.829348 13031 ts_tablet_manager.cc:1950] T ef062e1775bd4a3c859124eaaba81399 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:42.831496 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 75) took 1821 ms. Trace:
I20260926 19:58:42.831862 13031 rpcz_store.cc:276] 0926 19:58:41.010058 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:41.010164 (+ 106us) service_pool.cc:224] Handling call
0926 19:58:42.610607 (+1600443us) ts_tablet_manager.cc:1151] Deleting tablet ef062e1775bd4a3c859124eaaba81399
0926 19:58:42.610646 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:42.763580 (+152934us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:42.827757 (+ 64177us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:42.831382 (+ 3625us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1599438}
I20260926 19:58:42.833039 13012 tablet_bootstrap.cc:654] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.833156 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet ef062e1775bd4a3c859124eaaba81399 (table default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]) successfully deleted
I20260926 19:58:42.844895 13013 ts_tablet_manager.cc:1434] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.026s user 0.005s sys 0.001s
I20260926 19:58:42.855078 13013 tablet_bootstrap.cc:492] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:42.862491 13031 tablet_replica.cc:333] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:42.863211 13031 raft_consensus.cc:2243] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:42.864010 13031 pending_rounds.cc:70] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Trying to abort 1 pending ops.
I20260926 19:58:42.864220 13031 pending_rounds.cc:77] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694351343816704 op_type: NO_OP noop_request { }
I20260926 19:58:42.864651 13031 raft_consensus.cc:2889] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:42.864987 13031 raft_consensus.cc:2272] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.866693 13092 tablet_bootstrap.cc:654] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.870836 13013 tablet_bootstrap.cc:654] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:42.875128 13031 ts_tablet_manager.cc:1916] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.893450 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 552d11cb23144691809c554c125cad0e (DEFAULT_TABLE table=default.Table_35ed1349ab724c9182fdb237e2630b50 [id=0a3fcb2373e24226a1424faaa9083504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.893572 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 552d11cb23144691809c554c125cad0e (DEFAULT_TABLE table=default.Table_35ed1349ab724c9182fdb237e2630b50 [id=0a3fcb2373e24226a1424faaa9083504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.894520 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d11cb23144691809c554c125cad0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.894538 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d11cb23144691809c554c125cad0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.900036 12391 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet b82d75be43e841e7abcc9e9db800aa2d (table default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa]). Expected version 2 got 0
I20260926 19:58:42.900130 12902 tablet_service.cc:1511] Processing CreateTablet for tablet a5324b1b682f43d4a689b6fafff3df01 (DEFAULT_TABLE table=default.Table_b2cab18b22ac46d9bd64e47ac24851e0 [id=c6c677e3a26345b1bc7db3d6374cf578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.901073 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5324b1b682f43d4a689b6fafff3df01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.907490 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 552d11cb23144691809c554c125cad0e (DEFAULT_TABLE table=default.Table_35ed1349ab724c9182fdb237e2630b50 [id=0a3fcb2373e24226a1424faaa9083504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.908579 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 552d11cb23144691809c554c125cad0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.910979 12564 tablet_service.cc:1511] Processing CreateTablet for tablet a5324b1b682f43d4a689b6fafff3df01 (DEFAULT_TABLE table=default.Table_b2cab18b22ac46d9bd64e47ac24851e0 [id=c6c677e3a26345b1bc7db3d6374cf578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.911362 12737 tablet_service.cc:1511] Processing CreateTablet for tablet a5324b1b682f43d4a689b6fafff3df01 (DEFAULT_TABLE table=default.Table_b2cab18b22ac46d9bd64e47ac24851e0 [id=c6c677e3a26345b1bc7db3d6374cf578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.911904 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5324b1b682f43d4a689b6fafff3df01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.912328 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5324b1b682f43d4a689b6fafff3df01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.914368 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 7ef773d1c6694c63a011819024451b02 (DEFAULT_TABLE table=default.Table_5f027d31993e4d35a2937f694c90a59c [id=dfc3435de1e7453794433737b7c342fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.915457 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ef773d1c6694c63a011819024451b02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.915493 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 7ef773d1c6694c63a011819024451b02 (DEFAULT_TABLE table=default.Table_5f027d31993e4d35a2937f694c90a59c [id=dfc3435de1e7453794433737b7c342fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.916405 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ef773d1c6694c63a011819024451b02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.920754 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 7ef773d1c6694c63a011819024451b02 (DEFAULT_TABLE table=default.Table_5f027d31993e4d35a2937f694c90a59c [id=dfc3435de1e7453794433737b7c342fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.921711 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ef773d1c6694c63a011819024451b02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.923733 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 77b319b685344a848462b78617eb6c22 (DEFAULT_TABLE table=default.Table_a958288944144fbd9cc934add059cdb0 [id=1850b06b63d24da0b91ad1a54eae3dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.924664 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b319b685344a848462b78617eb6c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.929349 12734 tablet_service.cc:1511] Processing CreateTablet for tablet 77b319b685344a848462b78617eb6c22 (DEFAULT_TABLE table=default.Table_a958288944144fbd9cc934add059cdb0 [id=1850b06b63d24da0b91ad1a54eae3dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.930402 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b319b685344a848462b78617eb6c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.931084 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 16df0828847d4e7db5702d84082fa402 (DEFAULT_TABLE table=default.Table_29ce86fb5af04ec4b8411a4f3135b79d [id=74eae66d927e413a9457c4c89cff78ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.932157 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16df0828847d4e7db5702d84082fa402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.905239 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
I20260926 19:58:42.936864 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 16df0828847d4e7db5702d84082fa402 (DEFAULT_TABLE table=default.Table_29ce86fb5af04ec4b8411a4f3135b79d [id=74eae66d927e413a9457c4c89cff78ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.937970 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16df0828847d4e7db5702d84082fa402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.943636 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 8ff071a16f1244cebc8c93e5508280a3 (DEFAULT_TABLE table=default.Table_cb7930cd5c56467d9d6ccf644263f999 [id=e452740c55c345a9ab7445009c7a71cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.944658 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ff071a16f1244cebc8c93e5508280a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.951975 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 66c8916d88dd48998eab514bd3c538b9 (DEFAULT_TABLE table=default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.953082 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66c8916d88dd48998eab514bd3c538b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:42.963043 13092 tablet_bootstrap.cc:492] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:42.963575 13092 ts_tablet_manager.cc:1403] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.139s user 0.006s sys 0.006s
I20260926 19:58:42.964624 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet e550463602ff4b5f98e10dd593897bcf with delete_type TABLET_DATA_DELETED (replaced by tablet d48ee26c0aaa4abc8e08a1a2dadb028d) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:42.965251 13032 tablet_replica.cc:333] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:42.965997 13032 raft_consensus.cc:2243] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:42.966464 13032 raft_consensus.cc:2272] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:42.967106 13092 raft_consensus.cc:359] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.967777 13092 raft_consensus.cc:385] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:42.968052 13092 raft_consensus.cc:740] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:42.968156 13032 ts_tablet_manager.cc:1916] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:42.969484 12735 tablet_service.cc:1558] Processing DeleteTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:42.976536 13231 raft_consensus.cc:493] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:42.976984 13231 raft_consensus.cc:515] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:42.979046 13231 leader_election.cc:290] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:42.983412 12858 leader_election.cc:343] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 1d25f740c1334c5cb53d7cffa3eb275e
W20260926 19:58:42.984169 12857 leader_election.cc:343] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 1d25f740c1334c5cb53d7cffa3eb275e
I20260926 19:58:42.984532 12857 leader_election.cc:304] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:42.984874 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 8ff071a16f1244cebc8c93e5508280a3 (DEFAULT_TABLE table=default.Table_cb7930cd5c56467d9d6ccf644263f999 [id=e452740c55c345a9ab7445009c7a71cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.985366 13048 raft_consensus.cc:2749] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:42.990160 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f399698f9db84d8aaecd7ba4c9e96f07"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:42.992199 12394 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_35ed1349ab724c9182fdb237e2630b50" } new_table_name: "default.Table_935b6b15b7c944ab8b9714014acd40e8" modify_external_catalogs: true
W20260926 19:58:42.996133 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f399698f9db84d8aaecd7ba4c9e96f07 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:42.966431 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ea151df659944f9a094fe9410c3f4bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:42.997720 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 16df0828847d4e7db5702d84082fa402 (DEFAULT_TABLE table=default.Table_29ce86fb5af04ec4b8411a4f3135b79d [id=74eae66d927e413a9457c4c89cff78ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:42.998682 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16df0828847d4e7db5702d84082fa402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.001072 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 77b319b685344a848462b78617eb6c22 (DEFAULT_TABLE table=default.Table_a958288944144fbd9cc934add059cdb0 [id=1850b06b63d24da0b91ad1a54eae3dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.002107 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77b319b685344a848462b78617eb6c22. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:43.007957 13130 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:58:43.008366 13130 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
I20260926 19:58:43.008335 12389 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_3152a51e41624d969eea96ab90354999 [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 1 got 0
W20260926 19:58:43.008733 13130 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:58:43.009097 13130 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:58:43.009428 12389 catalog_manager.cc:5838] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:43.012840 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d25f740c1334c5cb53d7cffa3eb275e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:38 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.011292 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_4459dc1bf4e04cf6841f30e912b951ac" } new_table_name: "default.Table_2d039187041243c8adf2651cf58f0b4a" modify_external_catalogs: true
I20260926 19:58:43.027555 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 8ff071a16f1244cebc8c93e5508280a3 (DEFAULT_TABLE table=default.Table_cb7930cd5c56467d9d6ccf644263f999 [id=e452740c55c345a9ab7445009c7a71cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.028630 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ff071a16f1244cebc8c93e5508280a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.031560 13012 tablet_bootstrap.cc:492] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:43.031967 13012 ts_tablet_manager.cc:1403] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.279s user 0.007s sys 0.002s
I20260926 19:58:43.033326 13013 tablet_bootstrap.cc:492] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:43.033787 13013 ts_tablet_manager.cc:1403] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.179s user 0.000s sys 0.016s
I20260926 19:58:43.035467 13032 ts_tablet_manager.cc:1929] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:43.036834 13013 raft_consensus.cc:359] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.038750 13032 log.cc:1199] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/e550463602ff4b5f98e10dd593897bcf
I20260926 19:58:43.039557 13032 ts_tablet_manager.cc:1950] T e550463602ff4b5f98e10dd593897bcf P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:43.040659 13031 ts_tablet_manager.cc:1929] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:43.041023 13031 log.cc:1199] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/99cf26cf4838454c98aa8337341ffd06
I20260926 19:58:43.040726 13012 raft_consensus.cc:359] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.041343 13012 raft_consensus.cc:385] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.041625 13012 raft_consensus.cc:740] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:43.042338 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet e550463602ff4b5f98e10dd593897bcf (table default.Table_1ef063c29bb54a258c2650cf73e9abd0 [id=21d0c2efe78e438e8fad11d811c463ed]) successfully deleted
I20260926 19:58:43.042232 13012 consensus_queue.cc:260] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.044831 13012 ts_tablet_manager.cc:1434] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260926 19:58:43.057821 13031 ts_tablet_manager.cc:1950] T 99cf26cf4838454c98aa8337341ffd06 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:43.061906 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_1cd57fde79174a2e8092337be2e108c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:43.063582 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cd57fde79174a2e8092337be2e108c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:43.063690 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 74) took 2054 ms. Trace:
I20260926 19:58:43.063995 13031 rpcz_store.cc:276] 0926 19:58:41.009651 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:41.011415 (+ 1764us) service_pool.cc:224] Handling call
0926 19:58:42.861847 (+1850432us) ts_tablet_manager.cc:1151] Deleting tablet 99cf26cf4838454c98aa8337341ffd06
0926 19:58:42.861929 (+ 82us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:42.943201 (+ 81272us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:43.040540 (+ 97339us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:43.063626 (+ 23086us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1849865}
I20260926 19:58:43.065371 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 99cf26cf4838454c98aa8337341ffd06 (table default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]) successfully deleted
I20260926 19:58:43.066424 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 27ccbbe1a5784e97827a31e9febd8c20
I20260926 19:58:43.014456 13092 consensus_queue.cc:260] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.070909 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.078500 13092 ts_tablet_manager.cc:1434] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.115s user 0.005s sys 0.000s
I20260926 19:58:43.078677 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_87cff927dd5d4f0d9d947b7a5629728c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:43.080128 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87cff927dd5d4f0d9d947b7a5629728c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:43.082093 13092 tablet_bootstrap.cc:492] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:43.089628 13013 raft_consensus.cc:385] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.089906 13013 raft_consensus.cc:740] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:43.090636 13013 consensus_queue.cc:260] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.094061 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 80) took 1842 ms. Trace:
I20260926 19:58:43.094364 13031 rpcz_store.cc:276] 0926 19:58:41.251961 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:41.252495 (+ 534us) service_pool.cc:224] Handling call
0926 19:58:43.093691 (+1841196us) ts_tablet_manager.cc:1151] Deleting tablet b19f42511ad14d8a88fc0eaefb82e335
0926 19:58:43.093750 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:43.094008 (+ 258us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1840577}
I20260926 19:58:43.095520 13031 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 85) took 1543 ms. Trace:
I20260926 19:58:43.095661 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 886ab0feae1e49629290bc892e518816
I20260926 19:58:43.095815 13031 rpcz_store.cc:276] 0926 19:58:41.552297 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:41.557465 (+ 5168us) service_pool.cc:224] Handling call
0926 19:58:43.095197 (+1537732us) ts_tablet_manager.cc:1151] Deleting tablet e550463602ff4b5f98e10dd593897bcf
0926 19:58:43.095244 (+ 47us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:43.095474 (+ 230us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1537127}
I20260926 19:58:43.097013 12387 catalog_manager.cc:6934] ReplaceTablet: tablet 886ab0feae1e49629290bc892e518816 deleted and replaced by tablet 2ad76fc959e24545bdfad19f988fce93
I20260926 19:58:43.097324 13031 tablet_replica.cc:333] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:43.099236 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 886ab0feae1e49629290bc892e518816 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad76fc959e24545bdfad19f988fce93) from {username='slave'} at 127.0.0.1:33594
W20260926 19:58:43.099295 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet b19f42511ad14d8a88fc0eaefb82e335 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b19f42511ad14d8a88fc0eaefb82e335 already in progress: creating tablet
W20260926 19:58:43.099905 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet e550463602ff4b5f98e10dd593897bcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e550463602ff4b5f98e10dd593897bcf already in progress: creating tablet
W20260926 19:58:43.100466 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 56e337767be84507acdecfb189705bdd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56e337767be84507acdecfb189705bdd already in progress: creating tablet
I20260926 19:58:43.103734 13013 ts_tablet_manager.cc:1434] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.070s user 0.000s sys 0.005s
I20260926 19:58:43.104539 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 886ab0feae1e49629290bc892e518816 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad76fc959e24545bdfad19f988fce93) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:43.107196 13031 raft_consensus.cc:2243] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 19:58:43.110108 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 886ab0feae1e49629290bc892e518816 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 886ab0feae1e49629290bc892e518816 already in progress: creating tablet
I20260926 19:58:43.117873 13031 raft_consensus.cc:2272] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:43.117852 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 66c8916d88dd48998eab514bd3c538b9 (DEFAULT_TABLE table=default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.118929 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66c8916d88dd48998eab514bd3c538b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.120570 13212 raft_consensus.cc:493] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.121011 13212 raft_consensus.cc:515] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.123358 13212 leader_election.cc:290] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.124975 13031 ts_tablet_manager.cc:1916] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:43.128386 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea151df659944f9a094fe9410c3f4bb" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:43.131161 12565 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:43.131412 12521 leader_election.cc:343] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:43.132817 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 27ccbbe1a5784e97827a31e9febd8c20 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27ccbbe1a5784e97827a31e9febd8c20 already in progress: creating tablet
I20260926 19:58:43.136267 13013 tablet_bootstrap.cc:492] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:43.141280 13013 tablet_bootstrap.cc:654] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:43.150434 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 16897e7489e9445ba5389a4f3507a739 (DEFAULT_TABLE table=default.Table_d7b3b0491c1a417ea7d465c4662e7dee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.151654 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16897e7489e9445ba5389a4f3507a739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.152020 13212 raft_consensus.cc:493] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.152468 13212 raft_consensus.cc:515] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.154714 13212 leader_election.cc:290] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.061087 13032 tablet_replica.cc:333] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:43.154983 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:43.155696 13032 raft_consensus.cc:2243] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:43.156147 13032 raft_consensus.cc:2272] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:43.156885 12521 leader_election.cc:343] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:43.157817 13032 ts_tablet_manager.cc:1916] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:43.061290 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ff071a16f1244cebc8c93e5508280a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.069415 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.098522 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 886ab0feae1e49629290bc892e518816 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad76fc959e24545bdfad19f988fce93) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.126223 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea151df659944f9a094fe9410c3f4bb" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:43.161640 12760 raft_consensus.cc:2468] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:43.126755 13149 raft_consensus.cc:493] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.158002 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 16897e7489e9445ba5389a4f3507a739 (DEFAULT_TABLE table=default.Table_d7b3b0491c1a417ea7d465c4662e7dee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.162787 13149 raft_consensus.cc:515] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.163367 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16897e7489e9445ba5389a4f3507a739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.164773 13149 leader_election.cc:290] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:43.132748 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 66c8916d88dd48998eab514bd3c538b9 (DEFAULT_TABLE table=default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.166906 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e75eb4b3e8493b9d599c1c4ad4c67f" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:43.167316 12519 leader_election.cc:304] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:43.168040 13212 raft_consensus.cc:2804] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260926 19:58:43.168179 12696 leader_election.cc:343] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:43.168370 13212 raft_consensus.cc:493] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:43.168679 13212 raft_consensus.cc:3060] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:43.169063 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e75eb4b3e8493b9d599c1c4ad4c67f" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:43.169315 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 16897e7489e9445ba5389a4f3507a739 (DEFAULT_TABLE table=default.Table_d7b3b0491c1a417ea7d465c4662e7dee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.154567 13092 tablet_bootstrap.cc:654] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:43.170570 12695 leader_election.cc:343] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:43.171001 12695 leader_election.cc:304] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:43.172143 13149 raft_consensus.cc:2749] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:43.173837 12901 tablet_service.cc:1511] Processing CreateTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 (DEFAULT_TABLE table=default.Table_4459dc1bf4e04cf6841f30e912b951ac [id=dba288eafec847baaf3732ce6d4292b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.174935 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b734f0d33342d3be8b1827bcd5dd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.179044 13212 raft_consensus.cc:515] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.185323 12562 tablet_service.cc:1511] Processing CreateTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 (DEFAULT_TABLE table=default.Table_4459dc1bf4e04cf6841f30e912b951ac [id=dba288eafec847baaf3732ce6d4292b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.186474 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b734f0d33342d3be8b1827bcd5dd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.190055 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 0fd184e6295d47aeba809be6aab9ce1c
I20260926 19:58:43.191774 12393 catalog_manager.cc:6934] ReplaceTablet: tablet 0fd184e6295d47aeba809be6aab9ce1c deleted and replaced by tablet 22f268b4a1f7423f917731c6b62aab3e
I20260926 19:58:43.191977 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c with delete_type TABLET_DATA_DELETED (replaced by tablet 22f268b4a1f7423f917731c6b62aab3e) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.193578 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c with delete_type TABLET_DATA_DELETED (replaced by tablet 22f268b4a1f7423f917731c6b62aab3e) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.196082 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c with delete_type TABLET_DATA_DELETED (replaced by tablet 22f268b4a1f7423f917731c6b62aab3e) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:43.201543 13031 ts_tablet_manager.cc:1929] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260926 19:58:43.201915 13031 log.cc:1199] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/bdcecfe46c3447e493f1ab8b5eff3c62
W20260926 19:58:43.202255 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 0fd184e6295d47aeba809be6aab9ce1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fd184e6295d47aeba809be6aab9ce1c already in progress: creating tablet
I20260926 19:58:43.202584 13031 ts_tablet_manager.cc:1950] T bdcecfe46c3447e493f1ab8b5eff3c62 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:43.204344 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_3024aa79f7c748b9aceef193e0602546"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:43.205117 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet bdcecfe46c3447e493f1ab8b5eff3c62 (table default.Table_14c589443e094e22bc1c798043fd6d04 [id=01bf90b95d6c402a9dbdf3e5b3f760d6]) successfully deleted
W20260926 19:58:43.205467 12521 consensus_peers.cc:597] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260926 19:58:43.206113 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 3215bbd5be56438491a73925ac9ccbe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3215bbd5be56438491a73925ac9ccbe6 already in progress: creating tablet
W20260926 19:58:43.206513 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3024aa79f7c748b9aceef193e0602546 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:43.207396 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:43.209074 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66c8916d88dd48998eab514bd3c538b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:43.209685 12519 leader_election.cc:343] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:43.210075 12519 leader_election.cc:304] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:43.210752 13261 raft_consensus.cc:2749] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:43.210157 13012 tablet_bootstrap.cc:492] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:43.218657 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16897e7489e9445ba5389a4f3507a739. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:43.222620 12380 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 27ccbbe1a5784e97827a31e9febd8c20 (table default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1 [id=36e060a7ae90436d8c3917819816609c], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 27ccbbe1a5784e97827a31e9febd8c20 consensus state has no leader
I20260926 19:58:43.222916 13012 tablet_bootstrap.cc:654] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:43.224893 13212 leader_election.cc:290] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.226056 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea151df659944f9a094fe9410c3f4bb" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:43.226575 12761 raft_consensus.cc:3060] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:43.205487 13031 tablet_replica.cc:333] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:43.225178 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ea151df659944f9a094fe9410c3f4bb" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:43.232110 13031 raft_consensus.cc:2243] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:43.232654 13031 raft_consensus.cc:2272] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:43.232932 12521 leader_election.cc:343] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:43.232873 12761 raft_consensus.cc:2468] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:43.234239 13032 ts_tablet_manager.cc:1929] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:43.234149 12519 leader_election.cc:304] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:43.239938 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 16df0828847d4e7db5702d84082fa402
I20260926 19:58:43.240697 12734 tablet_service.cc:1511] Processing CreateTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 (DEFAULT_TABLE table=default.Table_4459dc1bf4e04cf6841f30e912b951ac [id=dba288eafec847baaf3732ce6d4292b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.241762 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5b734f0d33342d3be8b1827bcd5dd14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.242981 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 16df0828847d4e7db5702d84082fa402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.243384 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 16df0828847d4e7db5702d84082fa402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.234565 13032 log.cc:1199] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/b19f42511ad14d8a88fc0eaefb82e335
I20260926 19:58:43.244629 13032 ts_tablet_manager.cc:1950] T b19f42511ad14d8a88fc0eaefb82e335 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:43.244674 13031 ts_tablet_manager.cc:1916] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:43.247110 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet b19f42511ad14d8a88fc0eaefb82e335 (table default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95 [id=4692b08d69b344cdaf9226c62886da70]) successfully deleted
I20260926 19:58:43.248469 13032 tablet_replica.cc:333] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:43.249179 13032 raft_consensus.cc:2243] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:43.249614 13032 raft_consensus.cc:2272] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:43.250978 13261 raft_consensus.cc:2804] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:43.251240 13032 ts_tablet_manager.cc:1916] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:43.251431 13261 raft_consensus.cc:697] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:43.252171 13261 consensus_queue.cc:237] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.258249 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 16df0828847d4e7db5702d84082fa402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:43.259963 13231 raft_consensus.cc:493] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:43.260167 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 16df0828847d4e7db5702d84082fa402 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16df0828847d4e7db5702d84082fa402 already in progress: creating tablet
I20260926 19:58:43.260557 13231 raft_consensus.cc:515] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.262871 13231 leader_election.cc:290] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.263744 13128 consensus_queue.cc:1048] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:43.265997 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf4547f148f49b8943f4e1c7f2132a7" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
W20260926 19:58:43.273556 12394 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 552d11cb23144691809c554c125cad0e consensus state has no leader
I20260926 19:58:43.278961 13092 tablet_bootstrap.cc:492] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:43.265487 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf4547f148f49b8943f4e1c7f2132a7" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:43.283749 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 59d1955b12dd47899b8a226fe0e980ae (DEFAULT_TABLE table=default.Table_be184ee4abbf48999888ee786db3d2e7 [id=4ac26cbd7204431a903947e5f17c09b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.294894 13092 ts_tablet_manager.cc:1403] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.213s user 0.010s sys 0.000s
I20260926 19:58:43.295358 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d1955b12dd47899b8a226fe0e980ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.297917 12388 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b79b586799c14e0ab126d0dc3d462136" } modify_external_catalogs: true
I20260926 19:58:43.298535 12388 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b79b586799c14e0ab126d0dc3d462136" } modify_external_catalogs: true
I20260926 19:58:43.320781 13013 tablet_bootstrap.cc:492] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:43.321193 13013 ts_tablet_manager.cc:1403] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.185s user 0.008s sys 0.000s
I20260926 19:58:43.322682 12899 tablet_service.cc:1511] Processing CreateTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d (DEFAULT_TABLE table=default.Table_1ef063c29bb54a258c2650cf73e9abd0 [id=21d0c2efe78e438e8fad11d811c463ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.323808 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d48ee26c0aaa4abc8e08a1a2dadb028d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.327740 12394 master_service.cc:960] ReplaceTablet: received request to replace tablet da8875d46f7c46819ee67e650b098c27 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:43.330034 13231 raft_consensus.cc:493] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.330567 13231 raft_consensus.cc:515] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.332659 13231 leader_election.cc:290] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.329365 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 59d1955b12dd47899b8a226fe0e980ae (DEFAULT_TABLE table=default.Table_be184ee4abbf48999888ee786db3d2e7 [id=4ac26cbd7204431a903947e5f17c09b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.341868 13013 raft_consensus.cc:359] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.346101 12857 leader_election.cc:304] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:43.336650 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d75be43e841e7abcc9e9db800aa2d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:43.347409 13231 raft_consensus.cc:2749] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:43.338868 12564 tablet_service.cc:1511] Processing CreateTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d (DEFAULT_TABLE table=default.Table_1ef063c29bb54a258c2650cf73e9abd0 [id=21d0c2efe78e438e8fad11d811c463ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.349300 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d48ee26c0aaa4abc8e08a1a2dadb028d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.346913 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 46bdb479c5384d0891db66943ef263b2 (DEFAULT_TABLE table=default.Table_7d44b553bb6a405da5119c4cf46b4ce2 [id=8b1ac579f93c40d8a103acc8088ac9db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.328210 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 59d1955b12dd47899b8a226fe0e980ae (DEFAULT_TABLE table=default.Table_be184ee4abbf48999888ee786db3d2e7 [id=4ac26cbd7204431a903947e5f17c09b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.351189 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bdb479c5384d0891db66943ef263b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.357801 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d1955b12dd47899b8a226fe0e980ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.359774 13031 ts_tablet_manager.cc:1929] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:43.360146 13031 log.cc:1199] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/4e82262ea1c441808e9765aef911c903
I20260926 19:58:43.360841 13031 ts_tablet_manager.cc:1950] T 4e82262ea1c441808e9765aef911c903 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:43.297268 13092 raft_consensus.cc:359] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.361963 13092 raft_consensus.cc:385] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.362221 13092 raft_consensus.cc:740] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:43.362921 13092 consensus_queue.cc:260] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.365003 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 46bdb479c5384d0891db66943ef263b2 (DEFAULT_TABLE table=default.Table_7d44b553bb6a405da5119c4cf46b4ce2 [id=8b1ac579f93c40d8a103acc8088ac9db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.371695 12924 raft_consensus.cc:3060] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:43.372648 13092 ts_tablet_manager.cc:1434] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.077s user 0.005s sys 0.000s
I20260926 19:58:43.372745 12741 tablet_service.cc:1511] Processing CreateTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d (DEFAULT_TABLE table=default.Table_1ef063c29bb54a258c2650cf73e9abd0 [id=21d0c2efe78e438e8fad11d811c463ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.383466 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bdb479c5384d0891db66943ef263b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.385176 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d1955b12dd47899b8a226fe0e980ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.387905 13032 ts_tablet_manager.cc:1929] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:43.388288 13032 log.cc:1199] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/0ea151df659944f9a094fe9410c3f4bb
I20260926 19:58:43.388892 13032 ts_tablet_manager.cc:1950] T 0ea151df659944f9a094fe9410c3f4bb P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:43.390878 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d48ee26c0aaa4abc8e08a1a2dadb028d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.391322 13311 consensus_queue.cc:1048] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:43.390136 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 62390.8us
I20260926 19:58:43.418263 13013 raft_consensus.cc:385] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.418591 13013 raft_consensus.cc:740] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:43.418936 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c5b734f0d33342d3be8b1827bcd5dd14 consensus state has no leader
I20260926 19:58:43.419198 13013 consensus_queue.cc:260] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.421020 13013 ts_tablet_manager.cc:1434] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.081s user 0.003s sys 0.000s
I20260926 19:58:43.422266 13092 tablet_bootstrap.cc:492] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:43.432030 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 4e82262ea1c441808e9765aef911c903 (table default.Table_c7b5800561444fd7ad044784765fd2bb [id=68ea017f68a5467c8c1d1c1ea98f4701]) successfully deleted
I20260926 19:58:43.456512 13031 tablet_replica.cc:333] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:43.457255 13031 raft_consensus.cc:2243] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:43.456607 12396 catalog_manager.cc:5838] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:43.457716 13031 raft_consensus.cc:2272] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:43.457929 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:43.458534 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:43.462386 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 46bdb479c5384d0891db66943ef263b2 (DEFAULT_TABLE table=default.Table_7d44b553bb6a405da5119c4cf46b4ce2 [id=8b1ac579f93c40d8a103acc8088ac9db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.463366 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46bdb479c5384d0891db66943ef263b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.464372 13013 tablet_bootstrap.cc:492] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:43.481333 12390 master_service.cc:960] ReplaceTablet: received request to replace tablet 66c8916d88dd48998eab514bd3c538b9 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:43.484391 13231 raft_consensus.cc:493] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.489202 13231 raft_consensus.cc:515] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.491438 13231 leader_election.cc:290] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.499532 12388 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 724f067e1a6e4702a328bc4abe03ee27
I20260926 19:58:43.502568 12391 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f1e412eb951f446990776744666978fc" } modify_external_catalogs: true
I20260926 19:58:43.503075 12391 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f1e412eb951f446990776744666978fc" } modify_external_catalogs: true
I20260926 19:58:43.505677 13092 tablet_bootstrap.cc:654] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:43.506083 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82d75be43e841e7abcc9e9db800aa2d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:43.509457 13012 tablet_bootstrap.cc:492] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:43.509884 13012 ts_tablet_manager.cc:1403] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.300s user 0.012s sys 0.005s
I20260926 19:58:43.512143 13012 raft_consensus.cc:359] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.512724 13012 raft_consensus.cc:385] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.512909 12857 leader_election.cc:304] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:43.513228 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6d07cb4c954f23be80b315a5748adc" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:43.513826 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_be184ee4abbf48999888ee786db3d2e7" } new_table_name: "default.Table_e2009414abeb4160a53f5d44b2ade3a5" modify_external_catalogs: true
I20260926 19:58:43.521744 13031 ts_tablet_manager.cc:1916] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:43.512956 13012 raft_consensus.cc:740] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:43.534793 13012 consensus_queue.cc:260] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.536844 13012 ts_tablet_manager.cc:1434] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260926 19:58:43.538228 13092 tablet_bootstrap.cc:492] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:43.538646 13092 ts_tablet_manager.cc:1403] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.117s user 0.005s sys 0.004s
I20260926 19:58:43.544946 13012 tablet_bootstrap.cc:492] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:43.546060 13048 raft_consensus.cc:493] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.546481 13048 raft_consensus.cc:515] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.548018 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6d07cb4c954f23be80b315a5748adc" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:43.549016 13048 leader_election.cc:290] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.549286 13325 tablet.cc:1751] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:43.550194 12858 leader_election.cc:304] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:43.552656 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3f3931a71048fd874c1131d44f7b00" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:43.555907 13048 raft_consensus.cc:2749] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:43.557416 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_3ea5bfbeba4f471f9761aceba506dfa4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:43.559333 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ea5bfbeba4f471f9761aceba506dfa4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:43.559582 13231 raft_consensus.cc:2749] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:43.564644 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 0ea151df659944f9a094fe9410c3f4bb (table default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]) successfully deleted
W20260926 19:58:43.566242 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 27ccbbe1a5784e97827a31e9febd8c20 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27ccbbe1a5784e97827a31e9febd8c20 already in progress: creating tablet
I20260926 19:58:43.566756 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba3f3931a71048fd874c1131d44f7b00" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:43.567472 12760 raft_consensus.cc:2393] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:43.568508 12857 leader_election.cc:304] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:43.574175 13231 raft_consensus.cc:3060] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:43.576534 13012 tablet_bootstrap.cc:654] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:43.590451 13231 raft_consensus.cc:2749] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:43.592334 13092 raft_consensus.cc:359] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.606711 13092 raft_consensus.cc:385] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.607177 13092 raft_consensus.cc:740] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:43.607954 13092 consensus_queue.cc:260] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.610873 13013 tablet_bootstrap.cc:700] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee: Time spent opening tablet: real 0.142s user 0.003s sys 0.000s
I20260926 19:58:43.611472 13013 tablet_bootstrap.cc:654] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:43.615473 13092 ts_tablet_manager.cc:1434] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.077s user 0.005s sys 0.001s
I20260926 19:58:43.618957 13092 tablet_bootstrap.cc:492] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:43.626583 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3f85639b2411482d9062c58f0a8bdbb3" } new_table_name: "default.Table_582e94c0978641ddafbbf586413fffb5" modify_external_catalogs: true
I20260926 19:58:43.631686 13031 ts_tablet_manager.cc:1929] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:43.632063 13031 log.cc:1199] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/1d25f740c1334c5cb53d7cffa3eb275e
I20260926 19:58:43.632690 13031 ts_tablet_manager.cc:1950] T 1d25f740c1334c5cb53d7cffa3eb275e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:43.646903 13092 tablet_bootstrap.cc:654] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:43.648401 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 1d25f740c1334c5cb53d7cffa3eb275e (table default.Table_b1794633093d4c15bfaa7f160828ff99 [id=c6f1da949bf242ae87b8bbdc6c1d334a]) successfully deleted
W20260926 19:58:43.654685 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 27ccbbe1a5784e97827a31e9febd8c20 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27ccbbe1a5784e97827a31e9febd8c20 already in progress: creating tablet
W20260926 19:58:43.656306 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 886ab0feae1e49629290bc892e518816 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 886ab0feae1e49629290bc892e518816 already in progress: creating tablet
W20260926 19:58:43.659718 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 0fd184e6295d47aeba809be6aab9ce1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fd184e6295d47aeba809be6aab9ce1c already in progress: creating tablet
I20260926 19:58:43.665408 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet da8875d46f7c46819ee67e650b098c27
I20260926 19:58:43.665942 13261 raft_consensus.cc:493] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.666401 13261 raft_consensus.cc:515] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.667136 12394 catalog_manager.cc:6934] ReplaceTablet: tablet da8875d46f7c46819ee67e650b098c27 deleted and replaced by tablet c4d86fa6b40b4fc49c4fa67b63792831
I20260926 19:58:43.669548 13261 leader_election.cc:290] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.669914 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e550463602ff4b5f98e10dd593897bcf" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:43.670814 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 16df0828847d4e7db5702d84082fa402 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16df0828847d4e7db5702d84082fa402 already in progress: creating tablet
W20260926 19:58:43.671166 12521 leader_election.cc:343] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:43.672396 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet da8875d46f7c46819ee67e650b098c27 with delete_type TABLET_DATA_DELETED (replaced by tablet c4d86fa6b40b4fc49c4fa67b63792831) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.673843 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet da8875d46f7c46819ee67e650b098c27 with delete_type TABLET_DATA_DELETED (replaced by tablet c4d86fa6b40b4fc49c4fa67b63792831) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:43.674532 13203 tablet_replica.cc:333] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
W20260926 19:58:43.674631 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet da8875d46f7c46819ee67e650b098c27 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da8875d46f7c46819ee67e650b098c27 already in progress: creating tablet
I20260926 19:58:43.679534 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_97bd38b19cf042a98c9638eaf3084180"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:43.681109 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97bd38b19cf042a98c9638eaf3084180 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:43.690836 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:43.691495 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet b82d75be43e841e7abcc9e9db800aa2d (table default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa]). Expected version 2 got 0
I20260926 19:58:43.692301 13092 tablet_bootstrap.cc:492] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:43.692773 13092 ts_tablet_manager.cc:1403] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.074s user 0.005s sys 0.004s
W20260926 19:58:43.695665 12387 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59d1955b12dd47899b8a226fe0e980ae consensus state has no leader
I20260926 19:58:43.695411 13092 raft_consensus.cc:359] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.696061 13092 raft_consensus.cc:385] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.696332 13092 raft_consensus.cc:740] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:43.696420 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 66c8916d88dd48998eab514bd3c538b9
I20260926 19:58:43.697104 13092 consensus_queue.cc:260] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.697751 12390 catalog_manager.cc:6934] ReplaceTablet: tablet 66c8916d88dd48998eab514bd3c538b9 deleted and replaced by tablet 700d05149a934d5e8bec9cc41873ad3e
I20260926 19:58:43.701116 13203 raft_consensus.cc:2243] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:43.701953 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 66c8916d88dd48998eab514bd3c538b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 700d05149a934d5e8bec9cc41873ad3e) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.702355 13203 raft_consensus.cc:2272] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:43.704959 12391 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 1455b99a65ff44ff933825196708f485
W20260926 19:58:43.709134 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99465590a0e64b35b8a8f07e8777b524 consensus state has no leader
I20260926 19:58:43.708937 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 66c8916d88dd48998eab514bd3c538b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 700d05149a934d5e8bec9cc41873ad3e) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:43.710992 13092 ts_tablet_manager.cc:1434] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.018s user 0.003s sys 0.004s
W20260926 19:58:43.713162 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 66c8916d88dd48998eab514bd3c538b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66c8916d88dd48998eab514bd3c538b9 already in progress: creating tablet
I20260926 19:58:43.716396 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 1455b99a65ff44ff933825196708f485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.717609 13203 ts_tablet_manager.cc:1916] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:43.719650 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 886ab0feae1e49629290bc892e518816 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 886ab0feae1e49629290bc892e518816 already in progress: creating tablet
W20260926 19:58:43.722368 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 1455b99a65ff44ff933825196708f485 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1455b99a65ff44ff933825196708f485 already in progress: creating tablet
I20260926 19:58:43.726680 13092 tablet_bootstrap.cc:492] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:43.727520 13130 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:33143,127.11.233.253:37913,127.11.233.252:44243
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.253:37913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.233.253
--webserver_port=36963
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:58:43.730785 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ea151df659944f9a094fe9410c3f4bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:43.731974 13261 raft_consensus.cc:493] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.732465 13261 raft_consensus.cc:515] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.735276 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 1455b99a65ff44ff933825196708f485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:43.735602 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b19f42511ad14d8a88fc0eaefb82e335" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:43.736945 12521 leader_election.cc:343] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:43.738299 13261 leader_election.cc:290] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:43.738950 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:43.740217 13130 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260926 19:58:43.740451 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:43.742836 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_d8c097bf6d3e4346bdb4573b0393525b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:43.748229 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8c097bf6d3e4346bdb4573b0393525b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:43.746733 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a958288944144fbd9cc934add059cdb0" } modify_external_catalogs: true
W20260926 19:58:43.749884 12519 leader_election.cc:343] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: e550463602ff4b5f98e10dd593897bcf
I20260926 19:58:43.750309 12519 leader_election.cc:304] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:43.750308 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a958288944144fbd9cc934add059cdb0" } modify_external_catalogs: true
I20260926 19:58:43.751973 13261 raft_consensus.cc:2749] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:43.759907 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet da8875d46f7c46819ee67e650b098c27 with delete_type TABLET_DATA_DELETED (replaced by tablet c4d86fa6b40b4fc49c4fa67b63792831) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.760528 13032 tablet_replica.cc:333] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:43.761228 13032 raft_consensus.cc:2243] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:43.761714 13032 raft_consensus.cc:2272] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:43.762614 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet ffac209a6cd8440289ee7765c3c24b40 with delete_type TABLET_DATA_DELETED (replaced by tablet 189eaca5fb2b4b0d83a2873e0495e5ff) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.763316 13031 tablet_replica.cc:333] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:43.763979 13031 raft_consensus.cc:2243] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:43.764413 13031 raft_consensus.cc:2272] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:43.763937 12924 raft_consensus.cc:3060] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:43.765072 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 0fd184e6295d47aeba809be6aab9ce1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fd184e6295d47aeba809be6aab9ce1c already in progress: creating tablet
W20260926 19:58:43.765746 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 16df0828847d4e7db5702d84082fa402 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16df0828847d4e7db5702d84082fa402 already in progress: creating tablet
I20260926 19:58:43.766518 12922 raft_consensus.cc:3060] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:43.771812 13031 ts_tablet_manager.cc:1916] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:43.774386 13212 consensus_queue.cc:1048] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:43.787231 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 28b592db99c1495aafb555ff2a76a154 with delete_type TABLET_DATA_DELETED (replaced by tablet 0a7d5e580f5645ab95fa03d5f669dc32) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:43.790803 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet e550463602ff4b5f98e10dd593897bcf with delete_type TABLET_DATA_DELETED (replaced by tablet d48ee26c0aaa4abc8e08a1a2dadb028d) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:43.791817 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:43.793325 12382 consensus_peers.cc:597] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 -> Peer db2490adb6e84090b780f52470e580ab (127.11.233.253:37913): Couldn't send request to peer db2490adb6e84090b780f52470e580ab. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260926 19:58:43.792507 13314 consensus_queue.cc:1048] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:43.792666 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet da8875d46f7c46819ee67e650b098c27 with delete_type TABLET_DATA_DELETED (replaced by tablet c4d86fa6b40b4fc49c4fa67b63792831) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.802786 13261 consensus_queue.cc:1048] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:43.803164 12519 leader_election.cc:343] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: b19f42511ad14d8a88fc0eaefb82e335
I20260926 19:58:43.803555 12519 leader_election.cc:304] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:43.804237 13128 raft_consensus.cc:2749] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:43.805027 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 66c8916d88dd48998eab514bd3c538b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 700d05149a934d5e8bec9cc41873ad3e) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.815642 13032 ts_tablet_manager.cc:1916] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:43.818183 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 1455b99a65ff44ff933825196708f485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.821442 13092 tablet_bootstrap.cc:654] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:43.759619 13130 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260926 19:58:43.855260 13013 tablet_bootstrap.cc:492] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:43.855682 13013 ts_tablet_manager.cc:1403] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.391s user 0.010s sys 0.000s
I20260926 19:58:43.867311 13092 tablet_bootstrap.cc:492] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:43.867725 13092 ts_tablet_manager.cc:1403] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.141s user 0.010s sys 0.001s
I20260926 19:58:43.873169 13032 ts_tablet_manager.cc:1929] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:43.873296 13012 tablet_bootstrap.cc:492] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:43.873538 13032 log.cc:1199] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/da8875d46f7c46819ee67e650b098c27
I20260926 19:58:43.873677 13012 ts_tablet_manager.cc:1403] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.329s user 0.015s sys 0.005s
I20260926 19:58:43.874235 13032 ts_tablet_manager.cc:1950] T da8875d46f7c46819ee67e650b098c27 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:43.875917 13012 raft_consensus.cc:359] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.876507 13012 raft_consensus.cc:385] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.876626 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet da8875d46f7c46819ee67e650b098c27 (table default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]) successfully deleted
I20260926 19:58:43.876770 13012 raft_consensus.cc:740] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:43.877882 13031 ts_tablet_manager.cc:1929] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:43.878240 13031 log.cc:1199] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/ffac209a6cd8440289ee7765c3c24b40
I20260926 19:58:43.878933 13031 ts_tablet_manager.cc:1950] T ffac209a6cd8440289ee7765c3c24b40 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
W20260926 19:58:43.893005 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet da8875d46f7c46819ee67e650b098c27 because the tablet was not found. No further retry: Not found: Tablet not found: da8875d46f7c46819ee67e650b098c27
W20260926 19:58:43.894587 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 66c8916d88dd48998eab514bd3c538b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66c8916d88dd48998eab514bd3c538b9 already in progress: creating tablet
W20260926 19:58:43.896155 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 1455b99a65ff44ff933825196708f485 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1455b99a65ff44ff933825196708f485 already in progress: creating tablet
I20260926 19:58:43.902477 12741 tablet_service.cc:1511] Processing CreateTablet for tablet cd86f01d82d041f4852f83f2547e697f (DEFAULT_TABLE table=default.Table_87cff927dd5d4f0d9d947b7a5629728c [id=6a77f8956ec6491d915b40237d54a548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.903446 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd86f01d82d041f4852f83f2547e697f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.903524 12567 tablet_service.cc:1511] Processing CreateTablet for tablet cd86f01d82d041f4852f83f2547e697f (DEFAULT_TABLE table=default.Table_87cff927dd5d4f0d9d947b7a5629728c [id=6a77f8956ec6491d915b40237d54a548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.904433 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd86f01d82d041f4852f83f2547e697f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.909492 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_499a09d79630415996bc999c678ae00f" } new_table_name: "default.Table_c489e926efca4dafb56b6642f8404c2b" modify_external_catalogs: true
I20260926 19:58:43.915091 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 (DEFAULT_TABLE table=default.Table_06d7f301c3cb4d0e8d374df103fab2f2 [id=ab940466309d4545b5a3f15ecc020742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.915555 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:43.916136 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c5ef9e1bca427fb889cbb17f77f999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.916567 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 (DEFAULT_TABLE table=default.Table_06d7f301c3cb4d0e8d374df103fab2f2 [id=ab940466309d4545b5a3f15ecc020742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.917492 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c5ef9e1bca427fb889cbb17f77f999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.920192 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_d8c097bf6d3e4346bdb4573b0393525b" } new_table_name: "default.Table_c7d455485dd14d60806f7a48b45ce98b" modify_external_catalogs: true
I20260926 19:58:43.925134 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet da8875d46f7c46819ee67e650b098c27 with delete_type TABLET_DATA_DELETED (replaced by tablet c4d86fa6b40b4fc49c4fa67b63792831) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:43.925560 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (DEFAULT_TABLE table=default.Table_37d98bf446e04c2992e3325245a72122 [id=c010c3a0beb1406983b45e32c3e5640d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.925801 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f05b7ded126448f2acba6af9bc981450"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:43.926481 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.926209 12734 tablet_service.cc:1511] Processing CreateTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 (DEFAULT_TABLE table=default.Table_27371e4eb0cd4fa48aaef20d2db04fcc [id=f5d5844ddbeb4542b5e7b02d22800604]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:43.927145 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f05b7ded126448f2acba6af9bc981450 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:43.927675 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89c8cbb097a644218e0ca033a6fdb4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.929853 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_56393c731f9e44fe823755a7e91ccc3e" } modify_external_catalogs: true
I20260926 19:58:43.930300 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_56393c731f9e44fe823755a7e91ccc3e" } modify_external_catalogs: true
I20260926 19:58:43.934755 13013 raft_consensus.cc:359] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.935402 13013 raft_consensus.cc:385] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:43.935655 13013 raft_consensus.cc:740] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:43.935842 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 (DEFAULT_TABLE table=default.Table_27371e4eb0cd4fa48aaef20d2db04fcc [id=f5d5844ddbeb4542b5e7b02d22800604]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.936226 13013 consensus_queue.cc:260] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.936954 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89c8cbb097a644218e0ca033a6fdb4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.937623 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (DEFAULT_TABLE table=default.Table_37d98bf446e04c2992e3325245a72122 [id=c010c3a0beb1406983b45e32c3e5640d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.939085 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.946120 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 791b4ef9be004d7ead26591c50c6f405 (DEFAULT_TABLE table=default.Table_1b16c5189a4649b181b45752c6b81e4a [id=942153cf768c44949c0fb6b8b904b883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.946301 12559 tablet_service.cc:1511] Processing CreateTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d (DEFAULT_TABLE table=default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.947101 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 791b4ef9be004d7ead26591c50c6f405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.950309 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acdb4bd8cd58444cb1d6d4b5610de20d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.955044 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 791b4ef9be004d7ead26591c50c6f405 (DEFAULT_TABLE table=default.Table_1b16c5189a4649b181b45752c6b81e4a [id=942153cf768c44949c0fb6b8b904b883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:43.955963 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 791b4ef9be004d7ead26591c50c6f405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:43.961848 13149 raft_consensus.cc:493] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:43.962291 13149 raft_consensus.cc:515] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:43.964623 13149 leader_election.cc:290] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:43.966276 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:43.967046 12595 raft_consensus.cc:2468] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:43.969671 12394 master_service.cc:960] ReplaceTablet: received request to replace tablet 16897e7489e9445ba5389a4f3507a739 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:43.970472 12695 leader_election.cc:304] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:43.971784 13312 raft_consensus.cc:2804] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:43.972097 13312 raft_consensus.cc:493] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:43.972383 13312 raft_consensus.cc:3060] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:43.999315 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_e03d90b1b5074be1a523ccfba1bf7852"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:44.000815 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e03d90b1b5074be1a523ccfba1bf7852 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:44.000871 13149 raft_consensus.cc:493] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:44.001343 13149 raft_consensus.cc:515] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.005120 12388 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
I20260926 19:58:44.005204 13149 leader_election.cc:290] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:44.005869 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed45552637024bec99a76da079f352ec" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:44.006440 12595 raft_consensus.cc:2468] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:44.007572 12695 leader_election.cc:304] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:44.008358 13149 raft_consensus.cc:2804] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:44.008714 13149 raft_consensus.cc:493] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:44.009081 13149 raft_consensus.cc:3060] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:44.013993 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 77b319b685344a848462b78617eb6c22
I20260926 19:58:44.016304 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet 77b319b685344a848462b78617eb6c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:44.017339 13031 tablet_replica.cc:333] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:44.018029 13031 raft_consensus.cc:2243] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:44.018471 13031 raft_consensus.cc:2272] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:44.020782 13092 raft_consensus.cc:359] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:44.022373 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f666e1a8d214172bf5b752e9ec374bd has no consensus state
I20260926 19:58:44.022717 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:44.022728 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:44.024446 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed45552637024bec99a76da079f352ec" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:44.024904 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet ec6d07cb4c954f23be80b315a5748adc
W20260926 19:58:44.025957 12696 leader_election.cc:343] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:44.026932 12696 leader_election.cc:343] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:44.021374 13092 raft_consensus.cc:385] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.027766 13092 raft_consensus.cc:740] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.031466 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet ec6d07cb4c954f23be80b315a5748adc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:44.031419 12898 tablet_service.cc:1511] Processing CreateTablet for tablet cd86f01d82d041f4852f83f2547e697f (DEFAULT_TABLE table=default.Table_87cff927dd5d4f0d9d947b7a5629728c [id=6a77f8956ec6491d915b40237d54a548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.032903 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd86f01d82d041f4852f83f2547e697f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.034124 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 (DEFAULT_TABLE table=default.Table_06d7f301c3cb4d0e8d374df103fab2f2 [id=ab940466309d4545b5a3f15ecc020742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.035018 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c5ef9e1bca427fb889cbb17f77f999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.035893 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 (DEFAULT_TABLE table=default.Table_27371e4eb0cd4fa48aaef20d2db04fcc [id=f5d5844ddbeb4542b5e7b02d22800604]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:44.036471 12390 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 64149f9e4d384ddbb922291ac7de35b7 consensus state has no leader
I20260926 19:58:44.036767 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89c8cbb097a644218e0ca033a6fdb4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.044453 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 77b319b685344a848462b78617eb6c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:44.038195 13092 consensus_queue.cc:260] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.047693 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 791b4ef9be004d7ead26591c50c6f405 (DEFAULT_TABLE table=default.Table_1b16c5189a4649b181b45752c6b81e4a [id=942153cf768c44949c0fb6b8b904b883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.040410 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet ffac209a6cd8440289ee7765c3c24b40 (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]) successfully deleted
I20260926 19:58:44.048682 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 791b4ef9be004d7ead26591c50c6f405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.050011 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 77b319b685344a848462b78617eb6c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:44.049827 13312 raft_consensus.cc:515] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.050909 13092 ts_tablet_manager.cc:1434] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.183s user 0.004s sys 0.004s
I20260926 19:58:44.051836 13312 leader_election.cc:290] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:44.052026 12903 tablet_service.cc:1511] Processing CreateTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d (DEFAULT_TABLE table=default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.052935 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acdb4bd8cd58444cb1d6d4b5610de20d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.053632 13092 tablet_bootstrap.cc:492] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:44.051024 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (DEFAULT_TABLE table=default.Table_37d98bf446e04c2992e3325245a72122 [id=c010c3a0beb1406983b45e32c3e5640d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:44.059700 12521 consensus_peers.cc:597] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:44.063462 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:44.065333 13013 ts_tablet_manager.cc:1434] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.209s user 0.006s sys 0.001s
I20260926 19:58:44.066277 12595 raft_consensus.cc:3060] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:44.072553 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet ec6d07cb4c954f23be80b315a5748adc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:44.072494 13012 consensus_queue.cc:260] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.074549 13012 ts_tablet_manager.cc:1434] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.201s user 0.004s sys 0.001s
I20260926 19:58:44.076021 12564 tablet_service.cc:1558] Processing DeleteTablet for tablet ec6d07cb4c954f23be80b315a5748adc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:44.082710 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.084360 13092 tablet_bootstrap.cc:654] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.093886 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_c47706d1c57b4554a7019c4ddfa12a24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:44.094307 12738 tablet_service.cc:1511] Processing CreateTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d (DEFAULT_TABLE table=default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.095213 13032 tablet_replica.cc:333] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
W20260926 19:58:44.095330 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c47706d1c57b4554a7019c4ddfa12a24 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:44.095876 13032 raft_consensus.cc:2243] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:44.095886 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
W20260926 19:58:44.095995 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet da8875d46f7c46819ee67e650b098c27 because the tablet was not found. No further retry: Not found: Tablet not found: da8875d46f7c46819ee67e650b098c27
I20260926 19:58:44.096596 13032 raft_consensus.cc:2272] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:44.096575 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 77b319b685344a848462b78617eb6c22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77b319b685344a848462b78617eb6c22 already in progress: creating tablet
I20260926 19:58:44.093325 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:44.095229 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acdb4bd8cd58444cb1d6d4b5610de20d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.101864 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b638b85037f24946b493b98629cab24e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:44.107200 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b638b85037f24946b493b98629cab24e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:44.111529 13013 tablet_bootstrap.cc:492] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:44.123859 12735 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0583424ef64d9ea3beeb1c01ceafba (DEFAULT_TABLE table=default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.130676 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0583424ef64d9ea3beeb1c01ceafba (DEFAULT_TABLE table=default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.131819 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0583424ef64d9ea3beeb1c01ceafba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.140115 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 16897e7489e9445ba5389a4f3507a739
I20260926 19:58:44.141829 12394 catalog_manager.cc:6934] ReplaceTablet: tablet 16897e7489e9445ba5389a4f3507a739 deleted and replaced by tablet 986a28107bd54199bf2cda96978150c3
I20260926 19:58:44.141956 13012 tablet_bootstrap.cc:492] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:44.144861 12736 tablet_service.cc:1558] Processing DeleteTablet for tablet 16897e7489e9445ba5389a4f3507a739 with delete_type TABLET_DATA_DELETED (replaced by tablet 986a28107bd54199bf2cda96978150c3) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:44.146317 12564 tablet_service.cc:1558] Processing DeleteTablet for tablet 16897e7489e9445ba5389a4f3507a739 with delete_type TABLET_DATA_DELETED (replaced by tablet 986a28107bd54199bf2cda96978150c3) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:44.160297 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 22f268b4a1f7423f917731c6b62aab3e (DEFAULT_TABLE table=default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.161244 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f268b4a1f7423f917731c6b62aab3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.123829 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:44.181854 12696 leader_election.cc:343] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:44.186718 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 16897e7489e9445ba5389a4f3507a739 with delete_type TABLET_DATA_DELETED (replaced by tablet 986a28107bd54199bf2cda96978150c3) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:44.187220 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0583424ef64d9ea3beeb1c01ceafba (DEFAULT_TABLE table=default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.188167 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0583424ef64d9ea3beeb1c01ceafba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.191098 13013 tablet_bootstrap.cc:654] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.200006 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0583424ef64d9ea3beeb1c01ceafba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.201294 13032 ts_tablet_manager.cc:1916] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:44.209796 13203 ts_tablet_manager.cc:1929] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:44.210167 13203 log.cc:1199] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/da8875d46f7c46819ee67e650b098c27
I20260926 19:58:44.210853 13203 ts_tablet_manager.cc:1950] T da8875d46f7c46819ee67e650b098c27 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:44.212327 12595 raft_consensus.cc:2468] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:44.214295 12393 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:44.216656 12695 leader_election.cc:304] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:44.217813 13312 raft_consensus.cc:2804] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:44.218220 13312 raft_consensus.cc:697] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:44.218961 13312 consensus_queue.cc:237] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.223533 12736 tablet_service.cc:1558] Processing DeleteTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d0583424ef64d9ea3beeb1c01ceafba) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:44.228660 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5f027d31993e4d35a2937f694c90a59c" } new_table_name: "default.Table_159e9c03205e4e989b5e0c296bca3da0" modify_external_catalogs: true
I20260926 19:58:44.240110 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_97bd38b19cf042a98c9638eaf3084180" } modify_external_catalogs: true
I20260926 19:58:44.240618 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_97bd38b19cf042a98c9638eaf3084180" } modify_external_catalogs: true
I20260926 19:58:44.242390 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_12649125d95742bd829f8c1b62504ab6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:44.243796 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12649125d95742bd829f8c1b62504ab6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:44.250150 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 22f268b4a1f7423f917731c6b62aab3e (DEFAULT_TABLE table=default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.250401 13149 raft_consensus.cc:515] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.253346 12394 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5e21794a58a648c8a5806d4c535a8f37" } new_table_name: "default.Table_653e7d4c6df34da98c75d496cee46f27" modify_external_catalogs: true
I20260926 19:58:44.259500 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_06d7f301c3cb4d0e8d374df103fab2f2" } modify_external_catalogs: true
I20260926 19:58:44.260084 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_06d7f301c3cb4d0e8d374df103fab2f2" } modify_external_catalogs: true
I20260926 19:58:44.269310 12560 tablet_service.cc:1511] Processing CreateTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 (DEFAULT_TABLE table=default.Table_1cd57fde79174a2e8092337be2e108c1 [id=f15adbe1da374a8b9b507ff371c1286c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:44.270465 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7ef773d1c6694c63a011819024451b02 consensus state has no leader
I20260926 19:58:44.270771 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b32f8b0d480547a58ba5c6fed6483a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.296002 12897 tablet_service.cc:1511] Processing CreateTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 (DEFAULT_TABLE table=default.Table_1cd57fde79174a2e8092337be2e108c1 [id=f15adbe1da374a8b9b507ff371c1286c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.296002 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 22f268b4a1f7423f917731c6b62aab3e (DEFAULT_TABLE table=default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.301245 13012 tablet_bootstrap.cc:700] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent opening tablet: real 0.117s user 0.002s sys 0.000s
I20260926 19:58:44.301940 13012 tablet_bootstrap.cc:654] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:44.314420 13350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:44.323870 13013 tablet_bootstrap.cc:492] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:44.324411 13013 ts_tablet_manager.cc:1403] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.257s user 0.009s sys 0.001s
I20260926 19:58:44.326855 13013 raft_consensus.cc:359] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.335812 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f268b4a1f7423f917731c6b62aab3e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:44.342321 12380 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 886ab0feae1e49629290bc892e518816 (table default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 886ab0feae1e49629290bc892e518816 consensus state has no leader
I20260926 19:58:44.344198 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b32f8b0d480547a58ba5c6fed6483a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.342969 13031 ts_tablet_manager.cc:1916] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:44.347255 13092 tablet_bootstrap.cc:492] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:44.347640 13092 ts_tablet_manager.cc:1403] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.295s user 0.021s sys 0.011s
I20260926 19:58:44.353744 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f268b4a1f7423f917731c6b62aab3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.356930 13092 raft_consensus.cc:359] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:44.363298 12389 catalog_manager.cc:5838] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:44.365155 12389 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:44.368613 13092 raft_consensus.cc:385] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.368889 13092 raft_consensus.cc:740] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.369482 13092 consensus_queue.cc:260] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:44.371181 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:44.379796 13092 ts_tablet_manager.cc:1434] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.032s user 0.004s sys 0.002s
I20260926 19:58:44.380616 13092 tablet_bootstrap.cc:492] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:44.382282 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:44.389051 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]). Expected version 1 got 0
I20260926 19:58:44.399700 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet da8875d46f7c46819ee67e650b098c27 (table default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]) successfully deleted
I20260926 19:58:44.404001 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed45552637024bec99a76da079f352ec" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:44.410254 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 66c8916d88dd48998eab514bd3c538b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66c8916d88dd48998eab514bd3c538b9 already in progress: creating tablet
I20260926 19:58:44.412495 12393 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 52fd53cf4e8043e2820e2676a57c61a0
I20260926 19:58:44.416024 13092 tablet_bootstrap.cc:654] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:44.405282 12696 leader_election.cc:343] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:44.433143 13092 tablet_bootstrap.cc:492] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:44.433555 13092 ts_tablet_manager.cc:1403] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.053s user 0.007s sys 0.003s
I20260926 19:58:44.435909 13092 raft_consensus.cc:359] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.436563 13092 raft_consensus.cc:385] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.436864 13092 raft_consensus.cc:740] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.447013 13312 raft_consensus.cc:493] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:44.447546 13312 raft_consensus.cc:515] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.449608 13312 leader_election.cc:290] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:44.450451 13092 consensus_queue.cc:260] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.450904 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3215bbd5be56438491a73925ac9ccbe6" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:44.451018 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 5930402db43740b4b291c6c238ecf52c (DEFAULT_TABLE table=default.Table_a195c0af637546dc947815ce7a384b8b [id=56f80b8cc8eb47358c033dc894be8645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.452191 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5930402db43740b4b291c6c238ecf52c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:44.453840 12696 leader_election.cc:343] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:44.460397 13092 ts_tablet_manager.cc:1434] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.027s user 0.004s sys 0.001s
I20260926 19:58:44.460506 13032 ts_tablet_manager.cc:1929] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:44.461431 13092 tablet_bootstrap.cc:492] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:44.460593 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:44.464834 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad76fc959e24545bdfad19f988fce93 (DEFAULT_TABLE table=default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.466017 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad76fc959e24545bdfad19f988fce93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.467072 12737 tablet_service.cc:1511] Processing CreateTablet for tablet e27c5b06d33c43ab9a483e1fc9b18234 (DEFAULT_TABLE table=default.Table_3024aa79f7c748b9aceef193e0602546 [id=24959f9689ad494b9ab54ceabc4ba05d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.468154 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e27c5b06d33c43ab9a483e1fc9b18234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.473626 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:44.474471 12739 tablet_service.cc:1511] Processing CreateTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 (DEFAULT_TABLE table=default.Table_1cd57fde79174a2e8092337be2e108c1 [id=f15adbe1da374a8b9b507ff371c1286c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.485956 13092 tablet_bootstrap.cc:654] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.508695 13092 tablet_bootstrap.cc:492] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:44.509140 13092 ts_tablet_manager.cc:1403] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.000s
I20260926 19:58:44.477653 13032 log.cc:1199] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/ec6d07cb4c954f23be80b315a5748adc
I20260926 19:58:44.511597 13032 ts_tablet_manager.cc:1950] T ec6d07cb4c954f23be80b315a5748adc P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:44.487514 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 89870d2c5b0342c4bdf66474fa8d38fe (DEFAULT_TABLE table=default.Table_efa6ffdea9394a91a51887f7b9d3d37a [id=d4d54f45ea8f458387c1fe0232a34d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.511550 13092 raft_consensus.cc:359] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.497222 12733 tablet_service.cc:1511] Processing CreateTablet for tablet fcad0bbb2bb44338bd56f2547e70ab9c (DEFAULT_TABLE table=default.Table_f399698f9db84d8aaecd7ba4c9e96f07 [id=9908f61c35834c3e85cebb19da71fc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.512126 13092 raft_consensus.cc:385] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.512353 13092 raft_consensus.cc:740] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.513296 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b32f8b0d480547a58ba5c6fed6483a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.516032 13032 tablet_replica.cc:333] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:44.516755 13032 raft_consensus.cc:2243] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:44.517253 13032 raft_consensus.cc:2272] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:44.517827 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89870d2c5b0342c4bdf66474fa8d38fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.522296 13092 consensus_queue.cc:260] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.399084 13203 tablet_replica.cc:333] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:44.538306 13203 raft_consensus.cc:2243] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:44.422284 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad76fc959e24545bdfad19f988fce93 (DEFAULT_TABLE table=default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.460727 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 89870d2c5b0342c4bdf66474fa8d38fe (DEFAULT_TABLE table=default.Table_efa6ffdea9394a91a51887f7b9d3d37a [id=d4d54f45ea8f458387c1fe0232a34d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.539485 13092 ts_tablet_manager.cc:1434] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.030s user 0.004s sys 0.005s
W20260926 19:58:44.334133 12519 consensus_peers.cc:597] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ea151df659944f9a094fe9410c3f4bb. This is attempt 1: this message will repeat every 5th retry.
W20260926 19:58:44.542435 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 208423us
I20260926 19:58:44.549474 13092 tablet_bootstrap.cc:492] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:44.557343 13032 ts_tablet_manager.cc:1916] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:44.559929 12921 raft_consensus.cc:3060] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:44.561802 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet ec6d07cb4c954f23be80b315a5748adc (table default.Table_56393c731f9e44fe823755a7e91ccc3e [id=137dcda542e04a92938b739a96b65ea9]) successfully deleted
I20260926 19:58:44.565110 13092 tablet_bootstrap.cc:654] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.567490 13203 pending_rounds.cc:70] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:44.567914 13203 pending_rounds.cc:77] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694354459779072 op_type: NO_OP noop_request { }
I20260926 19:58:44.568557 13203 raft_consensus.cc:2889] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:44.569419 13203 raft_consensus.cc:2272] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:44.572229 13261 raft_consensus.cc:493] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:44.572685 13261 raft_consensus.cc:515] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.574484 13261 leader_election.cc:290] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:44.576231 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732a34f4894940b3889bab5f0c0891e0" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:44.577741 12521 leader_election.cc:343] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:44.578524 13031 ts_tablet_manager.cc:1929] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:44.578977 13031 log.cc:1199] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/28b592db99c1495aafb555ff2a76a154
I20260926 19:58:44.579802 13031 ts_tablet_manager.cc:1950] T 28b592db99c1495aafb555ff2a76a154 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:44.589437 13203 ts_tablet_manager.cc:1916] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:44.481367 12565 tablet_service.cc:1511] Processing CreateTablet for tablet e27c5b06d33c43ab9a483e1fc9b18234 (DEFAULT_TABLE table=default.Table_3024aa79f7c748b9aceef193e0602546 [id=24959f9689ad494b9ab54ceabc4ba05d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.491961 12562 tablet_service.cc:1511] Processing CreateTablet for tablet fcad0bbb2bb44338bd56f2547e70ab9c (DEFAULT_TABLE table=default.Table_f399698f9db84d8aaecd7ba4c9e96f07 [id=9908f61c35834c3e85cebb19da71fc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.505498 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 5930402db43740b4b291c6c238ecf52c (DEFAULT_TABLE table=default.Table_a195c0af637546dc947815ce7a384b8b [id=56f80b8cc8eb47358c033dc894be8645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.539417 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad76fc959e24545bdfad19f988fce93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.606204 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 06c5ef9e1bca427fb889cbb17f77f999
I20260926 19:58:44.539799 13013 raft_consensus.cc:385] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.607295 13013 raft_consensus.cc:740] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:44.609082 12394 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0a7d5e580f5645ab95fa03d5f669dc32 consensus state has no leader
I20260926 19:58:44.608466 13013 consensus_queue.cc:260] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.610368 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:44 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:44.613101 13013 ts_tablet_manager.cc:1434] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.288s user 0.007s sys 0.000s
I20260926 19:58:44.617645 13031 tablet_replica.cc:333] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:44.619653 13031 raft_consensus.cc:2243] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:44.620289 13031 raft_consensus.cc:2272] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:44.622711 13031 ts_tablet_manager.cc:1916] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:44.631475 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 16897e7489e9445ba5389a4f3507a739 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16897e7489e9445ba5389a4f3507a739 already in progress: creating tablet
I20260926 19:58:44.633042 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcad0bbb2bb44338bd56f2547e70ab9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.634029 13013 tablet_bootstrap.cc:492] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:44.635421 12389 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_3152a51e41624d969eea96ab90354999 [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 1 got 0
I20260926 19:58:44.613159 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89870d2c5b0342c4bdf66474fa8d38fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.644961 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732a34f4894940b3889bab5f0c0891e0" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:44.645538 12760 raft_consensus.cc:2468] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:44.660349 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:44 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:44.630746 13149 leader_election.cc:290] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:44.662153 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcad0bbb2bb44338bd56f2547e70ab9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.665786 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5930402db43740b4b291c6c238ecf52c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.675710 13013 tablet_bootstrap.cc:654] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.684652 13031 ts_tablet_manager.cc:1929] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:44.686281 13031 log.cc:1199] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/ec6d07cb4c954f23be80b315a5748adc
I20260926 19:58:44.687728 13031 ts_tablet_manager.cc:1950] T ec6d07cb4c954f23be80b315a5748adc P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:44.692732 13032 ts_tablet_manager.cc:1929] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:44.693084 13032 log.cc:1199] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/3215bbd5be56438491a73925ac9ccbe6
I20260926 19:58:44.693708 13032 ts_tablet_manager.cc:1950] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:44.700937 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 3215bbd5be56438491a73925ac9ccbe6 (table default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]) successfully deleted
I20260926 19:58:44.702584 12519 leader_election.cc:304] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:44.706228 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e27c5b06d33c43ab9a483e1fc9b18234. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:44.706727 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 06c5ef9e1bca427fb889cbb17f77f999 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06c5ef9e1bca427fb889cbb17f77f999 already in progress: creating tablet
I20260926 19:58:44.712940 13261 raft_consensus.cc:2804] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:44.713673 13261 raft_consensus.cc:493] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:44.714048 13261 raft_consensus.cc:3060] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:44.718808 13013 tablet_bootstrap.cc:492] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:44.719235 13013 ts_tablet_manager.cc:1403] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.085s user 0.009s sys 0.000s
I20260926 19:58:44.720127 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 28b592db99c1495aafb555ff2a76a154 (table default.Table_653e7d4c6df34da98c75d496cee46f27 [id=c2c079dc87ec480a82aa5b57b110da17]) successfully deleted
W20260926 19:58:44.721024 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 77b319b685344a848462b78617eb6c22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77b319b685344a848462b78617eb6c22 already in progress: creating tablet
I20260926 19:58:44.721558 13012 tablet_bootstrap.cc:492] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:44.722018 13012 ts_tablet_manager.cc:1403] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.581s user 0.008s sys 0.010s
I20260926 19:58:44.721776 13013 raft_consensus.cc:359] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.722361 13013 raft_consensus.cc:385] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.722607 13013 raft_consensus.cc:740] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.723279 13013 consensus_queue.cc:260] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.725365 13013 ts_tablet_manager.cc:1434] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260926 19:58:44.732045 13013 tablet_bootstrap.cc:492] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:44.737325 13013 tablet_bootstrap.cc:654] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:44.749608 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 407298us
I20260926 19:58:44.751102 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3215bbd5be56438491a73925ac9ccbe6" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:44.754837 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed45552637024bec99a76da079f352ec" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:44.755347 12595 raft_consensus.cc:3060] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:44.763767 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet ec6d07cb4c954f23be80b315a5748adc (table default.Table_56393c731f9e44fe823755a7e91ccc3e [id=137dcda542e04a92938b739a96b65ea9]) successfully deleted
W20260926 19:58:44.764711 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 16897e7489e9445ba5389a4f3507a739 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16897e7489e9445ba5389a4f3507a739 already in progress: creating tablet
I20260926 19:58:44.766582 12595 raft_consensus.cc:2468] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:44.766983 12900 tablet_service.cc:1511] Processing CreateTablet for tablet e27c5b06d33c43ab9a483e1fc9b18234 (DEFAULT_TABLE table=default.Table_3024aa79f7c748b9aceef193e0602546 [id=24959f9689ad494b9ab54ceabc4ba05d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.768066 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e27c5b06d33c43ab9a483e1fc9b18234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.767942 12897 tablet_service.cc:1511] Processing CreateTablet for tablet 5930402db43740b4b291c6c238ecf52c (DEFAULT_TABLE table=default.Table_a195c0af637546dc947815ce7a384b8b [id=56f80b8cc8eb47358c033dc894be8645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.769440 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5930402db43740b4b291c6c238ecf52c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.780011 13092 tablet_bootstrap.cc:492] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:44.780418 13092 ts_tablet_manager.cc:1403] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.231s user 0.041s sys 0.018s
I20260926 19:58:44.782581 13013 tablet_bootstrap.cc:492] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:44.782999 13013 ts_tablet_manager.cc:1403] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.000s
I20260926 19:58:44.782939 13092 raft_consensus.cc:359] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.783540 13092 raft_consensus.cc:385] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.783798 13092 raft_consensus.cc:740] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.784369 13092 consensus_queue.cc:260] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.785286 13013 raft_consensus.cc:359] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.788661 13013 raft_consensus.cc:385] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.788935 13013 raft_consensus.cc:740] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.789582 13013 consensus_queue.cc:260] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.794726 13092 ts_tablet_manager.cc:1434] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260926 19:58:44.796283 13092 tablet_bootstrap.cc:492] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:44.803364 13013 ts_tablet_manager.cc:1434] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260926 19:58:44.812108 13013 tablet_bootstrap.cc:492] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:44.815851 13092 tablet_bootstrap.cc:654] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.817210 13013 tablet_bootstrap.cc:654] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.830688 13092 tablet_bootstrap.cc:492] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:44.831089 13092 ts_tablet_manager.cc:1403] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.000s
W20260926 19:58:44.837167 12695 leader_election.cc:343] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:44.837893 12695 leader_election.cc:304] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:58:44.838718 12695 raft_consensus.cc:2686] T 3215bbd5be56438491a73925ac9ccbe6 P 3ae87963ed5e4d92a896bf4a73af4da9: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 19:58:44.840814 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet da8875d46f7c46819ee67e650b098c27 with delete_type TABLET_DATA_DELETED (replaced by tablet c4d86fa6b40b4fc49c4fa67b63792831) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:44.841436 13031 tablet_replica.cc:333] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:44.842113 13031 raft_consensus.cc:2243] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:44.842531 13031 raft_consensus.cc:2272] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:44.844189 13031 ts_tablet_manager.cc:1916] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:44.847357 13013 tablet_bootstrap.cc:492] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:44.847752 13013 ts_tablet_manager.cc:1403] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.000s
I20260926 19:58:44.849887 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:44.850667 13013 raft_consensus.cc:359] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.853080 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad76fc959e24545bdfad19f988fce93 (DEFAULT_TABLE table=default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.854030 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad76fc959e24545bdfad19f988fce93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.856482 13013 raft_consensus.cc:385] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.856738 13013 raft_consensus.cc:740] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.858242 13013 consensus_queue.cc:260] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.859045 13092 raft_consensus.cc:359] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.859635 13092 raft_consensus.cc:385] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.859889 13092 raft_consensus.cc:740] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.860459 13092 consensus_queue.cc:260] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.866863 13031 ts_tablet_manager.cc:1929] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:44.867199 13031 log.cc:1199] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/da8875d46f7c46819ee67e650b098c27
I20260926 19:58:44.867693 13013 ts_tablet_manager.cc:1434] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260926 19:58:44.868711 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 89870d2c5b0342c4bdf66474fa8d38fe (DEFAULT_TABLE table=default.Table_efa6ffdea9394a91a51887f7b9d3d37a [id=d4d54f45ea8f458387c1fe0232a34d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.869711 13092 ts_tablet_manager.cc:1434] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.038s user 0.008s sys 0.000s
I20260926 19:58:44.870116 12903 tablet_service.cc:1511] Processing CreateTablet for tablet fcad0bbb2bb44338bd56f2547e70ab9c (DEFAULT_TABLE table=default.Table_f399698f9db84d8aaecd7ba4c9e96f07 [id=9908f61c35834c3e85cebb19da71fc49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:44.870628 13092 tablet_bootstrap.cc:492] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:44.872761 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89870d2c5b0342c4bdf66474fa8d38fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.877555 13031 ts_tablet_manager.cc:1950] T da8875d46f7c46819ee67e650b098c27 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:44.877828 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcad0bbb2bb44338bd56f2547e70ab9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:44.888204 13261 raft_consensus.cc:515] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.891132 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732a34f4894940b3889bab5f0c0891e0" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:44.892383 12521 leader_election.cc:343] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:44.895505 13092 tablet_bootstrap.cc:654] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.903024 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:44 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:44.905416 13013 tablet_bootstrap.cc:492] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:44.905982 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_0718f84db29d46d08463ca1b53df6b71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:44.906584 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_e2009414abeb4160a53f5d44b2ade3a5" } new_table_name: "default.Table_ee77d6a00998425b94479cf721435bf0" modify_external_catalogs: true
I20260926 19:58:44.906826 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_7ce720ad66a14fc998927d063367314f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:44.908103 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0718f84db29d46d08463ca1b53df6b71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:44.908402 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ce720ad66a14fc998927d063367314f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:44.906179 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_447c2fd2552a48d0897da03fd0d1d963"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:44.911343 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_447c2fd2552a48d0897da03fd0d1d963 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:44.898597 13012 raft_consensus.cc:359] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.928349 13012 raft_consensus.cc:385] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:58:44.932833 12387 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59d1955b12dd47899b8a226fe0e980ae consensus state has no leader
I20260926 19:58:44.938787 13231 raft_consensus.cc:493] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:44.939237 13231 raft_consensus.cc:515] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.946298 13231 leader_election.cc:290] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:44.950157 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed45552637024bec99a76da079f352ec" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:44.952204 12595 raft_consensus.cc:2393] T ed45552637024bec99a76da079f352ec P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:44.958249 13374 tablet.cc:1751] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:44.964970 12396 master_service.cc:960] ReplaceTablet: received request to replace tablet 886ab0feae1e49629290bc892e518816 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:44.967206 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet da8875d46f7c46819ee67e650b098c27 (table default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]) successfully deleted
W20260926 19:58:44.967818 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 06c5ef9e1bca427fb889cbb17f77f999 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06c5ef9e1bca427fb889cbb17f77f999 already in progress: creating tablet
I20260926 19:58:44.969641 12396 master_service.cc:960] ReplaceTablet: received request to replace tablet acdb4bd8cd58444cb1d6d4b5610de20d from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:44.962126 13013 tablet_bootstrap.cc:654] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:44.961392 13373 tablet.cc:1751] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:44.974985 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed45552637024bec99a76da079f352ec" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:44.975342 12923 raft_consensus.cc:3060] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:44.975724 12760 raft_consensus.cc:2393] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:44.928653 13012 raft_consensus.cc:740] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.986194 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet b19f42511ad14d8a88fc0eaefb82e335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:44.986801 13261 leader_election.cc:290] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:44.917327 13203 ts_tablet_manager.cc:1929] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:44.988031 13203 log.cc:1199] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/0ea151df659944f9a094fe9410c3f4bb
I20260926 19:58:44.988709 13203 ts_tablet_manager.cc:1950] T 0ea151df659944f9a094fe9410c3f4bb P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:44.987540 13092 tablet_bootstrap.cc:492] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:44.990088 13092 ts_tablet_manager.cc:1403] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.120s user 0.015s sys 0.003s
I20260926 19:58:44.993983 13031 tablet_replica.cc:333] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:44.992350 13092 raft_consensus.cc:359] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:44.994506 13092 raft_consensus.cc:385] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:44.994765 13092 raft_consensus.cc:740] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:44.994831 13031 raft_consensus.cc:2243] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:44.995344 13031 raft_consensus.cc:2272] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:44.997361 13031 ts_tablet_manager.cc:1916] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:45.000396 13013 tablet_bootstrap.cc:492] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:45.000876 13013 ts_tablet_manager.cc:1403] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.097s user 0.010s sys 0.000s
I20260926 19:58:44.985179 13012 consensus_queue.cc:260] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.004618 12857 leader_election.cc:304] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:45.005635 13231 raft_consensus.cc:3060] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.007741 13092 consensus_queue.cc:260] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.015332 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732a34f4894940b3889bab5f0c0891e0" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:45.015890 12760 raft_consensus.cc:3060] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.022089 13013 raft_consensus.cc:359] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.022748 13013 raft_consensus.cc:385] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.023005 13013 raft_consensus.cc:740] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.026595 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 126) took 1296 ms. Trace:
I20260926 19:58:45.026926 13203 rpcz_store.cc:276] 0926 19:58:43.730550 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:43.730667 (+ 117us) service_pool.cc:224] Handling call
0926 19:58:44.398942 (+668275us) ts_tablet_manager.cc:1151] Deleting tablet 0ea151df659944f9a094fe9410c3f4bb
0926 19:58:44.398977 (+ 35us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:44.553980 (+155003us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b58000614d0. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a257a25da 00007f9a25824977 00007f9a2584eb45 00007f9a25c8f55c 00007f9a268f8c3a 00007f9a26929960 00007f9a2690fe6d 00007f9a2690fe19 00007f9a2690fda9 00007f9a2690fd71
0926 19:58:44.836312 (+282332us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:44.917210 (+ 80898us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:45.026536 (+109326us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12733,"spinlock_wait_cycles":30704384,"tablet-delete.queue_time_us":667764}
I20260926 19:58:45.028239 13203 tablet_replica.cc:333] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:45.028932 13203 raft_consensus.cc:2243] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:45.029323 13203 raft_consensus.cc:2272] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:45.031050 12923 raft_consensus.cc:1275] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Refusing update from remote peer 3ae87963ed5e4d92a896bf4a73af4da9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:45.033087 12760 raft_consensus.cc:2468] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:45.035553 13203 ts_tablet_manager.cc:1916] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:45.042268 13031 ts_tablet_manager.cc:1929] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:45.042611 13031 log.cc:1199] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/b19f42511ad14d8a88fc0eaefb82e335
I20260926 19:58:45.032245 13092 ts_tablet_manager.cc:1434] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.042s user 0.004s sys 0.002s
I20260926 19:58:45.043226 13031 ts_tablet_manager.cc:1950] T b19f42511ad14d8a88fc0eaefb82e335 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:45.043213 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_27371e4eb0cd4fa48aaef20d2db04fcc" } modify_external_catalogs: true
I20260926 19:58:45.043872 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_27371e4eb0cd4fa48aaef20d2db04fcc" } modify_external_catalogs: true
I20260926 19:58:45.046351 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_acdb314f8df24fea96383e9e16a5efe6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:45.049551 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acdb314f8df24fea96383e9e16a5efe6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:45.048835 13377 raft_consensus.cc:493] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.049969 13231 raft_consensus.cc:2749] T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.050185 13377 raft_consensus.cc:515] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.053207 13092 tablet_bootstrap.cc:492] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:45.057178 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet b19f42511ad14d8a88fc0eaefb82e335 (table default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95 [id=4692b08d69b344cdaf9226c62886da70]) successfully deleted
W20260926 19:58:45.066308 12381 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 64591.6us
I20260926 19:58:45.059427 13013 consensus_queue.cc:260] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.075307 13013 ts_tablet_manager.cc:1434] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.074s user 0.007s sys 0.000s
I20260926 19:58:45.076272 13013 tablet_bootstrap.cc:492] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:45.075809 13231 raft_consensus.cc:493] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.076206 13212 raft_consensus.cc:493] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.077558 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e75eb4b3e8493b9d599c1c4ad4c67f" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:45.078462 13212 raft_consensus.cc:515] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.078842 13231 raft_consensus.cc:515] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.080295 13212 leader_election.cc:290] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.078583 12595 raft_consensus.cc:2468] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:45.082098 13377 leader_election.cc:290] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.083498 12858 leader_election.cc:304] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:45.082919 13231 leader_election.cc:290] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.083496 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:45.084204 12595 raft_consensus.cc:2468] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:45.084966 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 0ea151df659944f9a094fe9410c3f4bb (table default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]) successfully deleted
I20260926 19:58:45.085363 12858 leader_election.cc:304] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:45.088409 13092 tablet_bootstrap.cc:654] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.089852 13377 raft_consensus.cc:2804] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:45.090282 13377 raft_consensus.cc:493] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:45.090675 13377 raft_consensus.cc:3060] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.094496 13379 raft_consensus.cc:2804] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 19:58:45.095029 13379 raft_consensus.cc:493] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:45.095506 13379 raft_consensus.cc:3060] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Advancing to term 2
W20260926 19:58:45.034504 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 197405us
I20260926 19:58:45.039935 13363 consensus_queue.cc:1048] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:45.101348 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:45.102563 12521 leader_election.cc:343] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.106050 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_21e4b470924b49518cf4ca4b8e336b36" } modify_external_catalogs: true
I20260926 19:58:45.106575 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_21e4b470924b49518cf4ca4b8e336b36" } modify_external_catalogs: true
I20260926 19:58:45.107095 13013 tablet_bootstrap.cc:654] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.108981 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_12649125d95742bd829f8c1b62504ab6" } new_table_name: "default.Table_cd8d867057984c05bb2d20afcbeebbe2" modify_external_catalogs: true
I20260926 19:58:45.102550 12695 leader_election.cc:304] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:45.114912 13363 raft_consensus.cc:2804] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:45.115347 13363 raft_consensus.cc:697] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:45.116072 13363 consensus_queue.cc:237] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.124202 13013 tablet_bootstrap.cc:492] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:45.125005 13013 ts_tablet_manager.cc:1403] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.049s user 0.010s sys 0.000s
I20260926 19:58:45.127305 13013 raft_consensus.cc:359] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.127919 13013 raft_consensus.cc:385] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.128167 13013 raft_consensus.cc:740] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.064584 13012 ts_tablet_manager.cc:1434] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.342s user 0.002s sys 0.005s
I20260926 19:58:45.129897 13012 tablet_bootstrap.cc:492] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:45.082587 13355 raft_consensus.cc:493] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.137749 12595 raft_consensus.cc:1275] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Refusing update from remote peer 3ae87963ed5e4d92a896bf4a73af4da9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:45.131754 13013 consensus_queue.cc:260] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.083371 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:45.098073 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e75eb4b3e8493b9d599c1c4ad4c67f" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:45.100158 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:45.145541 13013 ts_tablet_manager.cc:1434] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.020s user 0.005s sys 0.002s
I20260926 19:58:45.150520 13092 tablet_bootstrap.cc:492] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:45.151005 13092 ts_tablet_manager.cc:1403] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.098s user 0.007s sys 0.003s
I20260926 19:58:45.157071 13013 tablet_bootstrap.cc:492] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:45.157779 13311 consensus_queue.cc:1048] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260926 19:58:45.159523 13379 raft_consensus.cc:515] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.161201 12923 raft_consensus.cc:1240] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 2 FOLLOWER]: Rejecting Update request from peer 3ae87963ed5e4d92a896bf4a73af4da9 for earlier term 1. Current term is 2. Ops: [1.1-1.2]
I20260926 19:58:45.162489 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:45.163096 12595 raft_consensus.cc:3060] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:45.169960 13261 raft_consensus.cc:493] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.170826 13261 raft_consensus.cc:515] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.172533 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c47706d1c57b4554a7019c4ddfa12a24" } new_table_name: "default.Table_dcd1a07b1c9c420a911aeecc89dcc0da" modify_external_catalogs: true
I20260926 19:58:45.172794 13261 leader_election.cc:290] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.174439 13013 tablet_bootstrap.cc:654] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.175796 12595 raft_consensus.cc:2468] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 2.
I20260926 19:58:45.175760 12758 raft_consensus.cc:2468] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:45.176126 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9ad397471def442f87f3594e163fe1fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:45.176709 12594 raft_consensus.cc:1240] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 2 FOLLOWER]: Rejecting Update request from peer 3ae87963ed5e4d92a896bf4a73af4da9 for earlier term 1. Current term is 2. Ops: [1.1-1.2]
I20260926 19:58:45.176714 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27ccbbe1a5784e97827a31e9febd8c20" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:45.177664 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ad397471def442f87f3594e163fe1fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:45.178562 12519 leader_election.cc:343] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.179078 13311 consensus_queue.cc:1059] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:45.179828 13311 raft_consensus.cc:3055] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Stepping down as leader of term 1
I20260926 19:58:45.180125 13311 raft_consensus.cc:740] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:45.180943 12858 leader_election.cc:304] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:45.180847 13311 consensus_queue.cc:260] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.181115 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f07ac4e733db421bb4a3efc6f937c310"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:45.165547 13355 raft_consensus.cc:515] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:45.183008 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f07ac4e733db421bb4a3efc6f937c310 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:45.185832 13092 raft_consensus.cc:359] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.186483 13092 raft_consensus.cc:385] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.186748 13092 raft_consensus.cc:740] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.187291 12761 raft_consensus.cc:2468] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:45.187444 13092 consensus_queue.cc:260] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.129040 13377 raft_consensus.cc:515] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.186785 13355 leader_election.cc:290] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:45.190728 13377 leader_election.cc:290] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.191319 13377 raft_consensus.cc:2804] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:58:45.191534 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:45.191774 13377 raft_consensus.cc:697] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 2 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:45.191826 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_0911867d73d545f7bf86c97f052dee59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:45.192175 12594 raft_consensus.cc:2468] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
W20260926 19:58:45.193353 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0911867d73d545f7bf86c97f052dee59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:45.193360 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e75eb4b3e8493b9d599c1c4ad4c67f" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:45.194094 12595 raft_consensus.cc:3060] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.194537 13311 raft_consensus.cc:3060] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:45.197491 12695 leader_election.cc:304] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:45.200645 13363 raft_consensus.cc:2804] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:45.200989 13363 raft_consensus.cc:493] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:45.201326 13363 raft_consensus.cc:3060] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.210421 13379 leader_election.cc:290] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.209326 13203 ts_tablet_manager.cc:1929] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:45.217782 13203 log.cc:1199] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/1455b99a65ff44ff933825196708f485
I20260926 19:58:45.218497 13203 ts_tablet_manager.cc:1950] T 1455b99a65ff44ff933825196708f485 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:45.219687 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 125) took 1489 ms. Trace:
I20260926 19:58:45.220006 13203 rpcz_store.cc:276] 0926 19:58:43.729935 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:43.735136 (+ 5201us) service_pool.cc:224] Handling call
0926 19:58:45.028100 (+1292964us) ts_tablet_manager.cc:1151] Deleting tablet 1455b99a65ff44ff933825196708f485
0926 19:58:45.028138 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:45.110287 (+ 82149us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:45.209218 (+ 98931us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:45.219629 (+ 10411us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1292431}
I20260926 19:58:45.223735 13092 ts_tablet_manager.cc:1434] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.072s user 0.007s sys 0.000s
I20260926 19:58:45.227152 13092 tablet_bootstrap.cc:492] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:45.227738 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27ccbbe1a5784e97827a31e9febd8c20" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:45.228987 12521 leader_election.cc:343] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.229398 12521 leader_election.cc:304] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:45.230883 13261 raft_consensus.cc:2749] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.232916 12595 raft_consensus.cc:2468] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:45.232935 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:45.192586 13377 consensus_queue.cc:237] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.234185 12858 leader_election.cc:304] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:45.238845 13013 tablet_bootstrap.cc:492] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:45.239259 13013 ts_tablet_manager.cc:1403] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.093s user 0.007s sys 0.011s
I20260926 19:58:45.239627 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e75eb4b3e8493b9d599c1c4ad4c67f" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:45.240278 12757 raft_consensus.cc:3060] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.240489 13384 raft_consensus.cc:2804] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:45.245895 13013 raft_consensus.cc:359] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.246589 13013 raft_consensus.cc:385] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.246892 13013 raft_consensus.cc:740] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:45.252027 12696 leader_election.cc:343] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.252995 13092 tablet_bootstrap.cc:654] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.255540 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 1455b99a65ff44ff933825196708f485 (table default.Table_f1e412eb951f446990776744666978fc [id=bfb91cbd91534b4699fd4423b9923881]) successfully deleted
I20260926 19:58:45.221377 13203 tablet_replica.cc:333] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:45.257265 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2411a153e7194406a2bb7b879c08d7dd" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:45.258255 13203 raft_consensus.cc:2243] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:45.258754 13203 raft_consensus.cc:2272] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:45.260560 13203 ts_tablet_manager.cc:1916] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:45.261917 13013 consensus_queue.cc:260] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.263588 12519 leader_election.cc:304] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:45.265255 13012 tablet_bootstrap.cc:654] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.267072 13212 raft_consensus.cc:2804] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:45.267428 13212 raft_consensus.cc:493] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:45.267771 13212 raft_consensus.cc:3060] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.276098 13212 raft_consensus.cc:515] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.278775 13212 leader_election.cc:290] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.280670 13013 ts_tablet_manager.cc:1434] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.041s user 0.004s sys 0.003s
I20260926 19:58:45.282151 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:45.285013 12519 leader_election.cc:304] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:45.286152 13013 tablet_bootstrap.cc:492] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:45.289974 13212 raft_consensus.cc:2804] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:45.290797 13212 raft_consensus.cc:697] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:45.291694 13212 consensus_queue.cc:237] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:45.295493 12697 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s user 0.006s sys 0.013s
I20260926 19:58:45.247416 13384 raft_consensus.cc:697] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
W20260926 19:58:45.295861 12697 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s user 0.006s sys 0.014s
I20260926 19:58:45.296995 13384 consensus_queue.cc:237] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.280084 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:45.302392 12521 leader_election.cc:343] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.314628 12757 raft_consensus.cc:2468] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:45.323712 13092 tablet_bootstrap.cc:492] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:45.324126 13092 ts_tablet_manager.cc:1403] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.097s user 0.009s sys 0.000s
I20260926 19:58:45.334343 13013 tablet_bootstrap.cc:654] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.340658 13092 raft_consensus.cc:359] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:45.325419 12382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20260926 19:58:45.350276 12381 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 94766.1us
I20260926 19:58:45.341315 13092 raft_consensus.cc:385] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.351884 13092 raft_consensus.cc:740] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.352602 13092 consensus_queue.cc:260] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.358786 13092 ts_tablet_manager.cc:1434] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.034s user 0.004s sys 0.004s
I20260926 19:58:45.359772 13092 tablet_bootstrap.cc:492] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
W20260926 19:58:45.226958 13365 consensus_queue.cc:1175] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f6e9d6439fd54b28925a68a6145d3dee" responder_term: 2 status { last_received { term: 0 index: 0 } last_committed_idx: 0 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3ae87963ed5e4d92a896bf4a73af4da9 for earlier term 1. Current term is 2. Ops: [1.1-1.2]" } } last_received_current_leader { term: 0 index: 0 } }
I20260926 19:58:45.339838 13012 tablet_bootstrap.cc:492] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:45.365609 13012 ts_tablet_manager.cc:1403] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.236s user 0.011s sys 0.000s
I20260926 19:58:45.366142 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_3152a51e41624d969eea96ab90354999 [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 1 got 0
I20260926 19:58:45.368079 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_27b2665fcc774f07b2aaee9205ea4ad4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:45.368306 13012 raft_consensus.cc:359] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.368968 13012 raft_consensus.cc:385] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.370837 12758 raft_consensus.cc:2410] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260926 19:58:45.373590 13012 raft_consensus.cc:740] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.374331 13012 consensus_queue.cc:260] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.376410 13012 ts_tablet_manager.cc:1434] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260926 19:58:45.383142 12897 tablet_service.cc:1511] Processing CreateTablet for tablet edb035b50e8f445ba492e67796085123 (DEFAULT_TABLE table=default.Table_e03d90b1b5074be1a523ccfba1bf7852 [id=6ec8d0414f894264acc827003029b1f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.384375 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb035b50e8f445ba492e67796085123. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:45.369493 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27b2665fcc774f07b2aaee9205ea4ad4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:45.387748 13203 ts_tablet_manager.cc:1929] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:45.388114 13203 log.cc:1199] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/e550463602ff4b5f98e10dd593897bcf
W20260926 19:58:45.388110 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_3152a51e41624d969eea96ab90354999 [id=dd5c0a6b728948278a5dd48f8ec0bf6f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 62e75eb4b3e8493b9d599c1c4ad4c67f consensus state has no leader
I20260926 19:58:45.388715 13203 ts_tablet_manager.cc:1950] T e550463602ff4b5f98e10dd593897bcf P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:45.391284 12739 tablet_service.cc:1511] Processing CreateTablet for tablet edb035b50e8f445ba492e67796085123 (DEFAULT_TABLE table=default.Table_e03d90b1b5074be1a523ccfba1bf7852 [id=6ec8d0414f894264acc827003029b1f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.391867 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 6f666e1a8d214172bf5b752e9ec374bd (DEFAULT_TABLE table=default.Table_c7d455485dd14d60806f7a48b45ce98b [id=453a40bc60fd4eeaa2ffc14d887320ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.392340 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb035b50e8f445ba492e67796085123. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.392738 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f666e1a8d214172bf5b752e9ec374bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.362885 13363 raft_consensus.cc:515] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.396950 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 6f666e1a8d214172bf5b752e9ec374bd (DEFAULT_TABLE table=default.Table_c7d455485dd14d60806f7a48b45ce98b [id=453a40bc60fd4eeaa2ffc14d887320ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.398012 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f666e1a8d214172bf5b752e9ec374bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.408851 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 700d05149a934d5e8bec9cc41873ad3e (DEFAULT_TABLE table=default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.409972 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 700d05149a934d5e8bec9cc41873ad3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.411712 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 700d05149a934d5e8bec9cc41873ad3e (DEFAULT_TABLE table=default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.412823 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 700d05149a934d5e8bec9cc41873ad3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.414944 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 6f666e1a8d214172bf5b752e9ec374bd (DEFAULT_TABLE table=default.Table_c7d455485dd14d60806f7a48b45ce98b [id=453a40bc60fd4eeaa2ffc14d887320ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.415892 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f666e1a8d214172bf5b752e9ec374bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.417160 12394 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 64149f9e4d384ddbb922291ac7de35b7 (table default.Table_c489e926efca4dafb56b6642f8404c2b [id=00773dee014a47fca49e7dfd9ed0d584]). Expected version 2 got 0
I20260926 19:58:45.389366 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 128) took 1599 ms. Trace:
I20260926 19:58:45.418553 13012 tablet_bootstrap.cc:492] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:45.418443 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 700d05149a934d5e8bec9cc41873ad3e (DEFAULT_TABLE table=default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.419489 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 700d05149a934d5e8bec9cc41873ad3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.422276 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 59f7dc92330d400f93f427f3f79ad826 (DEFAULT_TABLE table=default.Table_c47706d1c57b4554a7019c4ddfa12a24 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.422286 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 986a28107bd54199bf2cda96978150c3 (DEFAULT_TABLE table=default.Table_d7b3b0491c1a417ea7d465c4662e7dee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.423197 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f7dc92330d400f93f427f3f79ad826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.423197 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986a28107bd54199bf2cda96978150c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.424467 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 59f7dc92330d400f93f427f3f79ad826 (DEFAULT_TABLE table=default.Table_c47706d1c57b4554a7019c4ddfa12a24 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.425269 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 986a28107bd54199bf2cda96978150c3 (DEFAULT_TABLE table=default.Table_d7b3b0491c1a417ea7d465c4662e7dee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.425419 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f7dc92330d400f93f427f3f79ad826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.426354 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986a28107bd54199bf2cda96978150c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.437220 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 986a28107bd54199bf2cda96978150c3 (DEFAULT_TABLE table=default.Table_d7b3b0491c1a417ea7d465c4662e7dee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.442046 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986a28107bd54199bf2cda96978150c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.442544 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 59f7dc92330d400f93f427f3f79ad826 (DEFAULT_TABLE table=default.Table_c47706d1c57b4554a7019c4ddfa12a24 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.443550 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59f7dc92330d400f93f427f3f79ad826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.439440 13092 tablet_bootstrap.cc:654] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.450233 12565 tablet_service.cc:1511] Processing CreateTablet for tablet edb035b50e8f445ba492e67796085123 (DEFAULT_TABLE table=default.Table_e03d90b1b5074be1a523ccfba1bf7852 [id=6ec8d0414f894264acc827003029b1f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.451220 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb035b50e8f445ba492e67796085123. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.455574 13013 tablet_bootstrap.cc:492] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:45.455971 13013 ts_tablet_manager.cc:1403] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.170s user 0.007s sys 0.008s
I20260926 19:58:45.456724 13092 tablet_bootstrap.cc:492] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:45.457127 13092 ts_tablet_manager.cc:1403] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.098s user 0.004s sys 0.004s
I20260926 19:58:45.458850 12394 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]). Expected version 2 got 0
I20260926 19:58:45.459481 13092 raft_consensus.cc:359] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.460088 13092 raft_consensus.cc:385] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.460341 13092 raft_consensus.cc:740] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.460503 12394 catalog_manager.cc:5838] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:45.469741 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 (DEFAULT_TABLE table=default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b [id=5639673ab1364e00a44bc7bce0428012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.470798 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fd4c88f9304f19883c3e5da8f937b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.471155 13012 tablet_bootstrap.cc:654] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.473629 13092 consensus_queue.cc:260] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.478677 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 (DEFAULT_TABLE table=default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b [id=5639673ab1364e00a44bc7bce0428012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.481078 12394 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
I20260926 19:58:45.481969 13384 raft_consensus.cc:493] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.482395 13384 raft_consensus.cc:515] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.484362 13384 leader_election.cc:290] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.488708 13092 ts_tablet_manager.cc:1434] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260926 19:58:45.488767 13212 raft_consensus.cc:493] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.489218 13212 raft_consensus.cc:515] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.490513 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 (DEFAULT_TABLE table=default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b [id=5639673ab1364e00a44bc7bce0428012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.490808 13092 tablet_bootstrap.cc:492] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:45.491528 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fd4c88f9304f19883c3e5da8f937b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.491416 13212 leader_election.cc:290] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.494531 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64149f9e4d384ddbb922291ac7de35b7" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:45.496095 12897 tablet_service.cc:1511] Processing CreateTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e (DEFAULT_TABLE table=default.Table_f05b7ded126448f2acba6af9bc981450 [id=ade55e2b356b4c9db57a1dd04b307ee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.497640 13013 raft_consensus.cc:359] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.498235 13013 raft_consensus.cc:385] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.498477 13013 raft_consensus.cc:740] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.496363 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64149f9e4d384ddbb922291ac7de35b7" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:45.479944 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fd4c88f9304f19883c3e5da8f937b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.500660 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e (DEFAULT_TABLE table=default.Table_f05b7ded126448f2acba6af9bc981450 [id=ade55e2b356b4c9db57a1dd04b307ee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.501694 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c7d0435f3564cb38b6cbca713009c2e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:45.501849 12521 leader_election.cc:343] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.502489 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c7d0435f3564cb38b6cbca713009c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.502314 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732a34f4894940b3889bab5f0c0891e0" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:45.499054 13013 consensus_queue.cc:260] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.510308 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "732a34f4894940b3889bab5f0c0891e0" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:45.511114 12758 raft_consensus.cc:2393] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:45.511497 13013 ts_tablet_manager.cc:1434] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.055s user 0.006s sys 0.001s
W20260926 19:58:45.511574 12387 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8f59be98ceef4effa15baf487c2d1140 consensus state has no leader
W20260926 19:58:45.512167 12519 leader_election.cc:343] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.512564 12519 leader_election.cc:304] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:45.512878 13013 tablet_bootstrap.cc:492] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:45.514322 13212 raft_consensus.cc:2749] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:45.518406 12390 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59f7dc92330d400f93f427f3f79ad826 consensus state has no leader
I20260926 19:58:45.518316 12857 leader_election.cc:304] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:45.519045 13377 raft_consensus.cc:3060] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.519834 12756 raft_consensus.cc:2393] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f6e9d6439fd54b28925a68a6145d3dee in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:45.525779 13377 raft_consensus.cc:2749] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.527549 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:45.530589 12696 leader_election.cc:343] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.536130 13013 tablet_bootstrap.cc:654] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.542667 13390 tablet.cc:1751] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:45.418474 13203 rpcz_store.cc:276] 0926 19:58:43.790222 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:43.790667 (+ 445us) service_pool.cc:224] Handling call
0926 19:58:45.221234 (+1430567us) ts_tablet_manager.cc:1151] Deleting tablet e550463602ff4b5f98e10dd593897bcf
0926 19:58:45.221274 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:45.328684 (+107410us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:45.387626 (+ 58942us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:45.389314 (+ 1688us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1429953}
I20260926 19:58:45.562597 13377 raft_consensus.cc:493] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.563074 13377 raft_consensus.cc:515] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.565021 13377 leader_election.cc:290] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:45.566761 12857 leader_election.cc:343] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 0ea151df659944f9a094fe9410c3f4bb
I20260926 19:58:45.544287 13363 leader_election.cc:290] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:45.619632 13012 tablet_bootstrap.cc:492] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:45.620031 13012 ts_tablet_manager.cc:1403] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.243s user 0.010s sys 0.002s
I20260926 19:58:45.621990 13203 tablet_replica.cc:333] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:45.625118 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet e550463602ff4b5f98e10dd593897bcf (table default.Table_1ef063c29bb54a258c2650cf73e9abd0 [id=21d0c2efe78e438e8fad11d811c463ed]) successfully deleted
I20260926 19:58:45.627393 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e (DEFAULT_TABLE table=default.Table_f05b7ded126448f2acba6af9bc981450 [id=ade55e2b356b4c9db57a1dd04b307ee1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.628186 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:45.628908 12595 raft_consensus.cc:2393] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:45.628904 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c7d0435f3564cb38b6cbca713009c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.633169 12740 tablet_service.cc:1511] Processing CreateTablet for tablet c4d86fa6b40b4fc49c4fa67b63792831 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.633684 12900 tablet_service.cc:1511] Processing CreateTablet for tablet c4d86fa6b40b4fc49c4fa67b63792831 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.634354 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4d86fa6b40b4fc49c4fa67b63792831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.634728 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4d86fa6b40b4fc49c4fa67b63792831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.641739 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 8f59be98ceef4effa15baf487c2d1140 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.641551 13012 raft_consensus.cc:359] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.642319 13012 raft_consensus.cc:385] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.642546 13012 raft_consensus.cc:740] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.642859 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f59be98ceef4effa15baf487c2d1140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.646867 13012 consensus_queue.cc:260] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.655913 12567 tablet_service.cc:1511] Processing CreateTablet for tablet c4d86fa6b40b4fc49c4fa67b63792831 (DEFAULT_TABLE table=default.Table_3a61927510f1455ead6604257dcf158c [id=7782d7d464814758beb7d276e88d6a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.655380 12695 leader_election.cc:304] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:45.660996 13363 raft_consensus.cc:2749] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.667367 13012 ts_tablet_manager.cc:1434] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.047s user 0.006s sys 0.000s
I20260926 19:58:45.670001 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4d86fa6b40b4fc49c4fa67b63792831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.671566 13012 tablet_bootstrap.cc:492] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:45.674728 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 700d05149a934d5e8bec9cc41873ad3e
I20260926 19:58:45.676224 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet 700d05149a934d5e8bec9cc41873ad3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:45.678292 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 89c8cbb097a644218e0ca033a6fdb4a2
I20260926 19:58:45.640059 13261 raft_consensus.cc:493] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.687493 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:45.693706 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:45.694530 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 700d05149a934d5e8bec9cc41873ad3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:45.694842 13012 tablet_bootstrap.cc:654] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:45.697784 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 89c8cbb097a644218e0ca033a6fdb4a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89c8cbb097a644218e0ca033a6fdb4a2 already in progress: creating tablet
I20260926 19:58:45.660578 12519 leader_election.cc:304] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
W20260926 19:58:45.702013 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 147030us
I20260926 19:58:45.708578 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet acdb4bd8cd58444cb1d6d4b5610de20d
I20260926 19:58:45.710209 12396 catalog_manager.cc:6934] ReplaceTablet: tablet acdb4bd8cd58444cb1d6d4b5610de20d deleted and replaced by tablet 7645307e0999482bbe3329c8d61cc567
I20260926 19:58:45.713635 13203 raft_consensus.cc:2243] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:45.714095 13203 raft_consensus.cc:2272] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:45.714207 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d with delete_type TABLET_DATA_DELETED (replaced by tablet 7645307e0999482bbe3329c8d61cc567) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:45.715638 13203 ts_tablet_manager.cc:1916] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:45.717367 12396 master_service.cc:960] ReplaceTablet: received request to replace tablet cd92a1a62ab24647bef7c9eefdf9c9d8 from {username='slave'} at 127.0.0.1:45734
W20260926 19:58:45.718133 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 700d05149a934d5e8bec9cc41873ad3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 700d05149a934d5e8bec9cc41873ad3e already in progress: creating tablet
I20260926 19:58:45.718333 13363 raft_consensus.cc:493] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:45.718906 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet acdb4bd8cd58444cb1d6d4b5610de20d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet acdb4bd8cd58444cb1d6d4b5610de20d already in progress: creating tablet
I20260926 19:58:45.718741 13363 raft_consensus.cc:515] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.720551 13363 leader_election.cc:290] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:45.738659 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64149f9e4d384ddbb922291ac7de35b7" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:45.739778 13092 tablet_bootstrap.cc:654] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:45.740149 12696 leader_election.cc:343] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.742852 13377 raft_consensus.cc:493] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.743711 13377 raft_consensus.cc:515] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.744454 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:45.744750 12732 tablet_service.cc:1511] Processing CreateTablet for tablet cc9f7a2e5d1c41a8ac793a31696fb311 (DEFAULT_TABLE table=default.Table_3ea5bfbeba4f471f9761aceba506dfa4 [id=87b4a0b0de7f410a8843a8c5e7ddee9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.745657 12904 tablet_service.cc:1511] Processing CreateTablet for tablet cc9f7a2e5d1c41a8ac793a31696fb311 (DEFAULT_TABLE table=default.Table_3ea5bfbeba4f471f9761aceba506dfa4 [id=87b4a0b0de7f410a8843a8c5e7ddee9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.745970 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9f7a2e5d1c41a8ac793a31696fb311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.745680 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 700d05149a934d5e8bec9cc41873ad3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:45.746572 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9f7a2e5d1c41a8ac793a31696fb311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.747196 13377 leader_election.cc:290] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.745831 12901 tablet_service.cc:1558] Processing DeleteTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d with delete_type TABLET_DATA_DELETED (replaced by tablet 7645307e0999482bbe3329c8d61cc567) from {username='slave'} at 127.0.0.1:33594
W20260926 19:58:45.755734 12857 leader_election.cc:343] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 3215bbd5be56438491a73925ac9ccbe6
I20260926 19:58:45.767970 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_49d5eefaa2aa4282b699143ac47a444d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:45.770267 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_1e6fa952aacb4642abb1628901cdb74f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:45.770417 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 8f59be98ceef4effa15baf487c2d1140 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.771652 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f59be98ceef4effa15baf487c2d1140. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:45.771960 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e6fa952aacb4642abb1628901cdb74f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:45.769440 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49d5eefaa2aa4282b699143ac47a444d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:45.774492 13377 raft_consensus.cc:493] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.774909 13377 raft_consensus.cc:515] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.777242 13377 leader_election.cc:290] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:45.780254 12857 leader_election.cc:343] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: e550463602ff4b5f98e10dd593897bcf
I20260926 19:58:45.787380 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_37d98bf446e04c2992e3325245a72122" } new_table_name: "default.Table_abfd9bc64fc246afb766ca3bd56518e5" modify_external_catalogs: true
I20260926 19:58:45.790130 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3152a51e41624d969eea96ab90354999" } new_table_name: "default.Table_13a9930424e74b249d88b4f42195f0ad" modify_external_catalogs: true
I20260926 19:58:45.795961 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_7a4372f262834b68bade3baafd4658b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:45.798606 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a4372f262834b68bade3baafd4658b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:45.808369 12353 master_runner.cc:187] Error getting master registration for 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111),
I20260926 19:58:45.808967 12353 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:58:45.688551 13261 raft_consensus.cc:515] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.812619 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d with delete_type TABLET_DATA_DELETED (replaced by tablet 7645307e0999482bbe3329c8d61cc567) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:45.814460 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 8f59be98ceef4effa15baf487c2d1140 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.814780 13261 leader_election.cc:290] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.813341 13212 raft_consensus.cc:2749] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.821398 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64149f9e4d384ddbb922291ac7de35b7" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:45.822013 12595 raft_consensus.cc:2468] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:45.825893 12695 leader_election.cc:304] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:45.827642 13363 raft_consensus.cc:2804] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:45.827994 13363 raft_consensus.cc:493] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:45.828390 13363 raft_consensus.cc:3060] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.830457 13092 tablet_bootstrap.cc:492] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:45.830942 13092 ts_tablet_manager.cc:1403] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.340s user 0.008s sys 0.010s
I20260926 19:58:45.833362 13092 raft_consensus.cc:359] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.834086 13092 raft_consensus.cc:385] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.834386 13092 raft_consensus.cc:740] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.835137 13092 consensus_queue.cc:260] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.835965 13363 raft_consensus.cc:515] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.851859 13013 tablet_bootstrap.cc:492] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:45.852290 13013 ts_tablet_manager.cc:1403] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.340s user 0.013s sys 0.007s
I20260926 19:58:45.853838 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f59be98ceef4effa15baf487c2d1140. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:45.865705 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 89c8cbb097a644218e0ca033a6fdb4a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89c8cbb097a644218e0ca033a6fdb4a2 already in progress: creating tablet
I20260926 19:58:45.867585 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3215bbd5be56438491a73925ac9ccbe6" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:45.868336 13398 raft_consensus.cc:493] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:45.868484 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9b2d4d45fdeb4802ac261b6c8de91c3c consensus state has no leader
W20260926 19:58:45.868722 12858 leader_election.cc:343] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 0ea151df659944f9a094fe9410c3f4bb
I20260926 19:58:45.868747 13398 raft_consensus.cc:515] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.868966 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189eaca5fb2b4b0d83a2873e0495e5ff" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:45.869091 12858 leader_election.cc:304] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:45.870020 13377 raft_consensus.cc:2749] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.870548 13398 leader_election.cc:290] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.871243 13092 ts_tablet_manager.cc:1434] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.040s user 0.005s sys 0.000s
W20260926 19:58:45.873688 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 122057us
I20260926 19:58:45.875514 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet cd92a1a62ab24647bef7c9eefdf9c9d8
I20260926 19:58:45.876899 12396 catalog_manager.cc:6934] ReplaceTablet: tablet cd92a1a62ab24647bef7c9eefdf9c9d8 deleted and replaced by tablet de85e3e17bf340f8895aff3f5421c85c
I20260926 19:58:45.879338 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189eaca5fb2b4b0d83a2873e0495e5ff" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:45.880694 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 with delete_type TABLET_DATA_DELETED (replaced by tablet de85e3e17bf340f8895aff3f5421c85c) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:45.883107 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a34ea809f74841d18b133415daf8192a" } modify_external_catalogs: true
I20260926 19:58:45.883288 13092 tablet_bootstrap.cc:492] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:45.883602 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a34ea809f74841d18b133415daf8192a" } modify_external_catalogs: true
I20260926 19:58:45.886451 12561 tablet_service.cc:1511] Processing CreateTablet for tablet cc9f7a2e5d1c41a8ac793a31696fb311 (DEFAULT_TABLE table=default.Table_3ea5bfbeba4f471f9761aceba506dfa4 [id=87b4a0b0de7f410a8843a8c5e7ddee9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.887550 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9f7a2e5d1c41a8ac793a31696fb311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.893378 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd184e6295d47aeba809be6aab9ce1c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:45.893589 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd92a1a62ab24647bef7c9eefdf9c9d8 already in progress: creating tablet
I20260926 19:58:45.894186 13363 leader_election.cc:290] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:58:45.895893 12519 leader_election.cc:343] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.897493 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64149f9e4d384ddbb922291ac7de35b7" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:45.897562 13013 raft_consensus.cc:359] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.898568 13013 raft_consensus.cc:385] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.898828 13013 raft_consensus.cc:740] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.898993 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd184e6295d47aeba809be6aab9ce1c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:45.899415 13013 consensus_queue.cc:260] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:45.900255 12696 leader_election.cc:343] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.900422 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64149f9e4d384ddbb922291ac7de35b7" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:45.900955 12593 raft_consensus.cc:3060] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.901307 13013 ts_tablet_manager.cc:1434] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.049s user 0.005s sys 0.000s
I20260926 19:58:45.911855 13092 tablet_bootstrap.cc:654] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.912053 12390 catalog_manager.cc:5838] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:45.914058 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_7ce720ad66a14fc998927d063367314f" } modify_external_catalogs: true
I20260926 19:58:45.918318 13377 raft_consensus.cc:493] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:45.918586 12858 leader_election.cc:343] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.918741 13377 raft_consensus.cc:515] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.918951 12858 leader_election.cc:304] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:45.919587 13384 raft_consensus.cc:2749] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:45.919890 12858 leader_election.cc:343] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: e550463602ff4b5f98e10dd593897bcf
I20260926 19:58:45.920224 12858 leader_election.cc:304] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:45.920715 13377 leader_election.cc:290] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:45.920783 13384 raft_consensus.cc:2749] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.914521 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_7ce720ad66a14fc998927d063367314f" } modify_external_catalogs: true
I20260926 19:58:45.930243 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:45.930953 12760 raft_consensus.cc:2393] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:45.933308 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 01d8740b03594023a8420c7a7121b80c (DEFAULT_TABLE table=default.Table_b638b85037f24946b493b98629cab24e [id=b748bf49932b4e0db7de11beebb15173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.933662 13012 tablet_bootstrap.cc:492] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:45.934399 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d8740b03594023a8420c7a7121b80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:45.941689 13012 ts_tablet_manager.cc:1403] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.270s user 0.023s sys 0.000s
I20260926 19:58:45.944021 13012 raft_consensus.cc:359] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.944638 13012 raft_consensus.cc:385] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:45.944880 13012 raft_consensus.cc:740] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:45.945636 13012 consensus_queue.cc:260] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.962720 13012 ts_tablet_manager.cc:1434] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260926 19:58:45.964120 13012 tablet_bootstrap.cc:492] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:45.965065 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 700d05149a934d5e8bec9cc41873ad3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 700d05149a934d5e8bec9cc41873ad3e already in progress: creating tablet
I20260926 19:58:45.967000 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 01d8740b03594023a8420c7a7121b80c (DEFAULT_TABLE table=default.Table_b638b85037f24946b493b98629cab24e [id=b748bf49932b4e0db7de11beebb15173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.969946 13013 tablet_bootstrap.cc:492] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
W20260926 19:58:45.906133 12521 leader_election.cc:343] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.918134 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 01d8740b03594023a8420c7a7121b80c (DEFAULT_TABLE table=default.Table_b638b85037f24946b493b98629cab24e [id=b748bf49932b4e0db7de11beebb15173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:45.922281 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:45.976837 12594 raft_consensus.cc:2393] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:45.977082 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d8740b03594023a8420c7a7121b80c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:45.934810 12519 leader_election.cc:343] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:45.977730 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet acdb4bd8cd58444cb1d6d4b5610de20d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet acdb4bd8cd58444cb1d6d4b5610de20d already in progress: creating tablet
I20260926 19:58:45.985646 12521 leader_election.cc:304] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:45.986332 13401 raft_consensus.cc:493] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:45.986826 13401 raft_consensus.cc:515] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:45.990147 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:45.991627 12519 leader_election.cc:343] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.992470 13013 tablet_bootstrap.cc:654] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:45.992813 13404 raft_consensus.cc:2749] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.993522 13401 leader_election.cc:290] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:45.994199 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 88095.1us
W20260926 19:58:45.995136 12521 leader_election.cc:343] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:45.978322 12858 leader_election.cc:304] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:45.995616 12521 leader_election.cc:304] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:45.996296 13384 raft_consensus.cc:3060] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:45.996706 13404 raft_consensus.cc:2749] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:46.003875 13384 raft_consensus.cc:2749] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:45.946530 13203 ts_tablet_manager.cc:1929] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:46.004427 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01d8740b03594023a8420c7a7121b80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:46.004962 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_498bd18aa31b447993f6ed4f665070a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:46.006270 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:46.007339 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_498bd18aa31b447993f6ed4f665070a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:45.951508 12593 raft_consensus.cc:2468] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:46.009295 13203 log.cc:1199] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/b19f42511ad14d8a88fc0eaefb82e335
I20260926 19:58:46.009248 12695 leader_election.cc:304] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:46.009997 13203 ts_tablet_manager.cc:1950] T b19f42511ad14d8a88fc0eaefb82e335 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:46.010313 13363 raft_consensus.cc:2804] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:46.010717 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 129) took 2220 ms. Trace:
I20260926 19:58:46.010993 13203 rpcz_store.cc:276] 0926 19:58:43.790537 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:43.791700 (+ 1163us) service_pool.cc:224] Handling call
0926 19:58:45.553447 (+1761747us) ts_tablet_manager.cc:1151] Deleting tablet b19f42511ad14d8a88fc0eaefb82e335
0926 19:58:45.553489 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:45.789160 (+235671us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:45.946410 (+157250us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:46.010668 (+ 64258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90851,"tablet-delete.queue_time_us":1761233}
I20260926 19:58:46.012053 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 with delete_type TABLET_DATA_DELETED (replaced by tablet de85e3e17bf340f8895aff3f5421c85c) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:46.012197 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet b19f42511ad14d8a88fc0eaefb82e335 (table default.Table_2c8a7b7277874c12a4c5c05e0cb3ad95 [id=4692b08d69b344cdaf9226c62886da70]) successfully deleted
I20260926 19:58:45.990525 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 with delete_type TABLET_DATA_DELETED (replaced by tablet de85e3e17bf340f8895aff3f5421c85c) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.014889 13363 raft_consensus.cc:697] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:46.015681 13363 consensus_queue.cc:237] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:46.017719 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd92a1a62ab24647bef7c9eefdf9c9d8 already in progress: creating tablet
I20260926 19:58:46.024912 13012 tablet_bootstrap.cc:654] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:46.036419 13092 tablet_bootstrap.cc:492] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:46.037205 13092 ts_tablet_manager.cc:1403] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.154s user 0.011s sys 0.000s
I20260926 19:58:46.040349 13092 raft_consensus.cc:359] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:46.046058 12521 leader_election.cc:343] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:46.050308 12396 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 7a12b8fbf62f42e58aabcae5bbd60026
I20260926 19:58:46.050647 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 136) took 2007 ms. Trace:
I20260926 19:58:46.051009 13203 rpcz_store.cc:276] 0926 19:58:44.043567 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:44.044297 (+ 730us) service_pool.cc:224] Handling call
0926 19:58:46.050311 (+2006014us) ts_tablet_manager.cc:1151] Deleting tablet 77b319b685344a848462b78617eb6c22
0926 19:58:46.050353 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:46.050591 (+ 238us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2005374}
I20260926 19:58:46.046463 12521 leader_election.cc:304] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:46.052558 13401 raft_consensus.cc:2749] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:46.052495 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 77b319b685344a848462b78617eb6c22 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77b319b685344a848462b78617eb6c22 already in progress: creating tablet
I20260926 19:58:46.053339 13203 tablet_replica.cc:333] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:46.054113 13203 raft_consensus.cc:2243] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:46.054661 13203 raft_consensus.cc:2272] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:46.056449 13203 ts_tablet_manager.cc:1916] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:46.052428 12387 catalog_manager.cc:5838] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 reported cstate change: term changed from 1 to 2, leader changed from 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194) to 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195). New cstate: current_term: 2 leader_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:46.058907 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:46.059661 12387 catalog_manager.cc:5838] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195). New cstate: current_term: 1 leader_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:46.060797 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_3152a51e41624d969eea96ab90354999 [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 1 got 0
I20260926 19:58:46.061463 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
W20260926 19:58:46.062896 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62e75eb4b3e8493b9d599c1c4ad4c67f consensus state has no leader
I20260926 19:58:46.065676 13092 raft_consensus.cc:385] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:46.066047 13092 raft_consensus.cc:740] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:46.066959 13092 consensus_queue.cc:260] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.087625 13092 ts_tablet_manager.cc:1434] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.050s user 0.004s sys 0.001s
I20260926 19:58:46.072649 13401 raft_consensus.cc:493] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:46.093158 13401 raft_consensus.cc:515] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.095347 13401 leader_election.cc:290] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:46.095182 12390 catalog_manager.cc:5838] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:46.086411 13013 tablet_bootstrap.cc:492] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:46.096374 13013 ts_tablet_manager.cc:1403] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.127s user 0.008s sys 0.002s
I20260926 19:58:46.096395 12390 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 64149f9e4d384ddbb922291ac7de35b7 (table default.Table_c489e926efca4dafb56b6642f8404c2b [id=00773dee014a47fca49e7dfd9ed0d584]). Expected version 2 got 0
I20260926 19:58:46.096354 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd92a1a62ab24647bef7c9eefdf9c9d8" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:46.098131 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet ba3f3931a71048fd874c1131d44f7b00
I20260926 19:58:46.097975 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd92a1a62ab24647bef7c9eefdf9c9d8" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:46.098776 13013 raft_consensus.cc:359] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.099395 13013 raft_consensus.cc:385] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:58:46.099642 12519 leader_election.cc:343] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:46.101141 12521 leader_election.cc:343] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:46.101481 12521 leader_election.cc:304] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:46.103178 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet ba3f3931a71048fd874c1131d44f7b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:46.105378 13012 tablet_bootstrap.cc:492] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:46.105669 13401 raft_consensus.cc:2749] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:46.105816 13012 ts_tablet_manager.cc:1403] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.142s user 0.009s sys 0.002s
W20260926 19:58:46.105896 12382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.253:37913, user_credentials={real_user=slave}} blocked reactor thread for 84431.4us
I20260926 19:58:46.109735 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet ba3f3931a71048fd874c1131d44f7b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.110419 13403 tablet_replica.cc:333] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:46.111182 13403 raft_consensus.cc:2243] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:46.111681 13403 raft_consensus.cc:2272] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:46.115687 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_cc8a261b07da4eb384b6ac40b7c5c5c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:46.116466 13401 consensus_queue.cc:1048] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:46.117331 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc8a261b07da4eb384b6ac40b7c5c5c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.118105 13384 consensus_queue.cc:1048] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:46.119220 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c with delete_type TABLET_DATA_DELETED (replaced by tablet 22f268b4a1f7423f917731c6b62aab3e) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:46.120744 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_e70be55e31fe48c8a83266f2d0937cab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.122264 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e70be55e31fe48c8a83266f2d0937cab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.124491 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet ba3f3931a71048fd874c1131d44f7b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:46.125694 13400 tablet_replica.cc:333] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:46.126550 13400 raft_consensus.cc:2243] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:46.127095 13400 raft_consensus.cc:2272] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:46.130690 13400 ts_tablet_manager.cc:1916] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:46.134148 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:46.136142 12390 master_service.cc:960] ReplaceTablet: received request to replace tablet cd86f01d82d041f4852f83f2547e697f from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:46.138247 13403 ts_tablet_manager.cc:1916] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:46.138989 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a87359d2c6874eacaee4a94038793c5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.140396 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a87359d2c6874eacaee4a94038793c5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:46.148475 12387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 50) took 753 ms (client timeout 1000 ms). Trace:
I20260926 19:58:46.152256 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ea151df659944f9a094fe9410c3f4bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:39 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.152671 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet e550463602ff4b5f98e10dd593897bcf with delete_type TABLET_DATA_DELETED (replaced by tablet d48ee26c0aaa4abc8e08a1a2dadb028d) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.154290 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1ef063c29bb54a258c2650cf73e9abd0" } new_table_name: "default.Table_1e071d1293234cecb72d6a118e8d5f10" modify_external_catalogs: true
I20260926 19:58:46.160647 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:46.161337 13409 consensus_queue.cc:1048] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:46.148857 12387 rpcz_store.cc:269] 0926 19:58:45.395342 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:45.512217 (+116875us) service_pool.cc:224] Handling call
0926 19:58:45.597818 (+ 85601us) spinlock_profiling.cc:243] Waited 83.8 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:46.148422 (+550604us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:46.063375 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:46.063857 (+ 482us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:46.063880 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:58:46.063893 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:46.064191 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:46.064226 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:46.064248 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:46.064505 (+ 257us) tablet.cc:701] Row locks acquired
0926 19:58:46.064518 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:46.064623 (+ 105us) write_op.cc:270] Start()
0926 19:58:46.064700 (+ 77us) write_op.cc:276] Timestamp: P: 1790452726064590 usec, L: 0
0926 19:58:46.064715 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:46.065147 (+ 432us) log.cc:844] Serialized 919 byte log entry
0926 19:58:46.083023 (+ 17876us) op_driver.cc:464] REPLICATION: finished
0926 19:58:46.085009 (+ 1986us) write_op.cc:301] APPLY: starting
0926 19:58:46.085058 (+ 49us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:46.085154 (+ 96us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:46.085240 (+ 86us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:46.085457 (+ 217us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:46.085633 (+ 176us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:58:46.085674 (+ 41us) write_op.cc:312] APPLY: finished
0926 19:58:46.085833 (+ 159us) log.cc:844] Serialized 29 byte log entry
0926 19:58:46.085966 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:46.086026 (+ 60us) write_op.cc:454] Released schema lock
0926 19:58:46.086054 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:46.086249 (+ 195us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:46.086273 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8657,"reported_tablets":6,"spinlock_wait_cycles":192634112,"child_traces":[["op",{"apply.queue_time_us":1869,"apply.run_cpu_time_us":1193,"apply.run_wall_time_us":1211,"num_ops":2,"prepare.queue_time_us":100,"prepare.run_cpu_time_us":1936,"prepare.run_wall_time_us":3013,"raft.queue_time_us":15922,"raft.run_cpu_time_us":18,"raft.run_wall_time_us":19,"replication_time_us":18235}]]}
W20260926 19:58:46.175987 12519 consensus_peers.cc:597] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:46.186220 13409 consensus_queue.cc:1048] T ed45552637024bec99a76da079f352ec P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:46.194496 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d48ee26c0aaa4abc8e08a1a2dadb028d consensus state has no leader
I20260926 19:58:46.201464 13400 ts_tablet_manager.cc:1929] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:46.202087 13400 log.cc:1199] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/ba3f3931a71048fd874c1131d44f7b00
I20260926 19:58:46.202960 13400 ts_tablet_manager.cc:1950] T ba3f3931a71048fd874c1131d44f7b00 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:46.206965 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]) successfully deleted
I20260926 19:58:46.208016 13400 tablet_replica.cc:333] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:46.208242 13012 raft_consensus.cc:359] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.213388 13012 raft_consensus.cc:385] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:46.213698 13012 raft_consensus.cc:740] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:46.215973 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet cd86f01d82d041f4852f83f2547e697f
I20260926 19:58:46.217252 12390 catalog_manager.cc:6934] ReplaceTablet: tablet cd86f01d82d041f4852f83f2547e697f deleted and replaced by tablet 3488f0076b0f4ef19f7cf23a93cd20e3
I20260926 19:58:46.218416 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet cd86f01d82d041f4852f83f2547e697f with delete_type TABLET_DATA_DELETED (replaced by tablet 3488f0076b0f4ef19f7cf23a93cd20e3) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:46.220287 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet cd86f01d82d041f4852f83f2547e697f with delete_type TABLET_DATA_DELETED (replaced by tablet 3488f0076b0f4ef19f7cf23a93cd20e3) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:46.208897 13400 raft_consensus.cc:2243] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:46.221056 13400 raft_consensus.cc:2272] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:46.223573 13400 ts_tablet_manager.cc:1916] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:46.230229 13092 tablet_bootstrap.cc:492] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:46.239611 12899 tablet_service.cc:1558] Processing DeleteTablet for tablet cd86f01d82d041f4852f83f2547e697f with delete_type TABLET_DATA_DELETED (replaced by tablet 3488f0076b0f4ef19f7cf23a93cd20e3) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.099648 13013 raft_consensus.cc:740] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:46.242627 13013 consensus_queue.cc:260] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.247064 13013 ts_tablet_manager.cc:1434] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.150s user 0.007s sys 0.000s
I20260926 19:58:46.260280 12899 tablet_service.cc:1558] Processing DeleteTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d0583424ef64d9ea3beeb1c01ceafba) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.260334 12594 raft_consensus.cc:1275] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Refusing update from remote peer 3ae87963ed5e4d92a896bf4a73af4da9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:58:46.261641 12899 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c with delete_type TABLET_DATA_DELETED (replaced by tablet 22f268b4a1f7423f917731c6b62aab3e) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.262137 13409 consensus_queue.cc:1048] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:46.284665 12696 consensus_peers.cc:597] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:46.289896 13384 consensus_queue.cc:1048] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:46.295188 13400 ts_tablet_manager.cc:1929] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:46.296409 13400 log.cc:1199] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/27ccbbe1a5784e97827a31e9febd8c20
I20260926 19:58:46.310359 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f399698f9db84d8aaecd7ba4c9e96f07" } new_table_name: "default.Table_e8b7e8464d3d4efda2843dfb2dff6da1" modify_external_catalogs: true
I20260926 19:58:46.316197 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_e4d93f2b15784ea8b7c7bdd2522017e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.318094 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4d93f2b15784ea8b7c7bdd2522017e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.319875 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a87359d2c6874eacaee4a94038793c5b" } modify_external_catalogs: true
I20260926 19:58:46.320462 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a87359d2c6874eacaee4a94038793c5b" } modify_external_catalogs: true
I20260926 19:58:46.321979 13424 tablet.cc:1751] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:46.324913 12758 raft_consensus.cc:1275] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 2 FOLLOWER]: Refusing update from remote peer 8112c0b43d9c4e5389aa3a18f18b58a5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260926 19:58:46.325572 13425 tablet.cc:1751] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:46.325408 12758 pending_rounds.cc:85] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting all ops after (but not including) 0
I20260926 19:58:46.326508 12758 pending_rounds.cc:107] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260926 19:58:46.326869 12758 raft_consensus.cc:2889] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260926 19:58:46.327276 12758 pending_rounds.cc:107] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting uncommitted ALTER_SCHEMA_OP operation due to leader change: 1.2
W20260926 19:58:46.327850 12758 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38310: Aborted: Op aborted by new leader
I20260926 19:58:46.328450 12758 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:38310 (request call id 147) took 1684 ms. Trace:
I20260926 19:58:46.333586 13400 ts_tablet_manager.cc:1950] T 27ccbbe1a5784e97827a31e9febd8c20 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:46.337539 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_2d039187041243c8adf2651cf58f0b4a" } modify_external_catalogs: true
I20260926 19:58:46.328830 12758 rpcz_store.cc:276] 0926 19:58:44.643683 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:44.646310 (+ 2627us) service_pool.cc:224] Handling call
0926 19:58:46.328382 (+1682072us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:44.969912 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0926 19:58:44.970304 (+ 392us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0926 19:58:44.970335 (+ 31us) alter_schema_op.cc:66] Acquired schema lock
0926 19:58:44.970380 (+ 45us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0926 19:58:44.970495 (+ 115us) alter_schema_op.cc:128] START. Timestamp: 1790452724970437
0926 19:58:44.970515 (+ 20us) op_driver.cc:348] REPLICATION: starting
0926 19:58:44.971313 (+ 798us) log.cc:844] Serialized 173 byte log entry
0926 19:58:46.327696 (+1356383us) op_driver.cc:464] REPLICATION: finished
0926 19:58:46.327745 (+ 49us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0926 19:58:46.327779 (+ 34us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":320058,"prepare.run_cpu_time_us":1525,"prepare.run_wall_time_us":4708,"raft.queue_time_us":1368,"raft.run_cpu_time_us":1467,"raft.run_wall_time_us":1468,"replication_time_us":1357035,"thread_start_us":326503,"threads_started":1}]]}
I20260926 19:58:46.340807 13384 consensus_queue.cc:1048] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260926 19:58:46.339254 12380 catalog_manager.cc:5142] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): alter failed for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]): Aborted: Op aborted by new leader
I20260926 19:58:46.347972 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 27ccbbe1a5784e97827a31e9febd8c20 (table default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1 [id=36e060a7ae90436d8c3917819816609c]) successfully deleted
W20260926 19:58:46.348694 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet cd86f01d82d041f4852f83f2547e697f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd86f01d82d041f4852f83f2547e697f already in progress: creating tablet
I20260926 19:58:46.349493 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_2d039187041243c8adf2651cf58f0b4a" } modify_external_catalogs: true
I20260926 19:58:46.373066 12396 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet a3da723e393e47cdbeff8ea821bded34
W20260926 19:58:46.374114 12390 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fcad0bbb2bb44338bd56f2547e70ab9c consensus state has no leader
I20260926 19:58:46.381817 13409 raft_consensus.cc:493] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:46.382264 13409 raft_consensus.cc:515] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.384146 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet c5b734f0d33342d3be8b1827bcd5dd14
I20260926 19:58:46.385344 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb593ce2cd4147febebfe442de9fdd28" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:46.388422 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:46.389668 12696 leader_election.cc:343] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:46.393714 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet c5b734f0d33342d3be8b1827bcd5dd14 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5b734f0d33342d3be8b1827bcd5dd14 already in progress: creating tablet
W20260926 19:58:46.406764 12381 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet d48ee26c0aaa4abc8e08a1a2dadb028d (table default.Table_1e071d1293234cecb72d6a118e8d5f10 [id=21d0c2efe78e438e8fad11d811c463ed], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d48ee26c0aaa4abc8e08a1a2dadb028d consensus state has no leader
I20260926 19:58:46.407851 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:46.397148 13012 consensus_queue.cc:260] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.411999 13012 ts_tablet_manager.cc:1434] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.306s user 0.005s sys 0.001s
W20260926 19:58:46.415422 13418 log.cc:927] Time spent T ed45552637024bec99a76da079f352ec P 8112c0b43d9c4e5389aa3a18f18b58a5: Append to log took a long time: real 0.150s user 0.000s sys 0.001s
I20260926 19:58:46.458693 12388 master_service.cc:960] ReplaceTablet: received request to replace tablet b32f8b0d480547a58ba5c6fed6483a74 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:46.461387 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_4db2758ba1704aa58c9aeeaccfc8a825"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.462862 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4db2758ba1704aa58c9aeeaccfc8a825 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.466118 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b2591dd15fb0435683b013e66d1ec83a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.467571 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2591dd15fb0435683b013e66d1ec83a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.467651 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_dcd1a07b1c9c420a911aeecc89dcc0da" } new_table_name: "default.Table_85695893a5b24687959b83a8360d1443" modify_external_catalogs: true
I20260926 19:58:46.473415 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f7366985fc9a4e2386af9d2847ab0424"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.474984 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7366985fc9a4e2386af9d2847ab0424 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.476619 12899 tablet_service.cc:1558] Processing DeleteTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:33594
W20260926 19:58:46.482534 12387 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59f7dc92330d400f93f427f3f79ad826 consensus state has no leader
I20260926 19:58:46.484638 13409 leader_election.cc:290] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:46.489437 13012 tablet_bootstrap.cc:492] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:46.502738 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet b32f8b0d480547a58ba5c6fed6483a74
I20260926 19:58:46.504124 12388 catalog_manager.cc:6934] ReplaceTablet: tablet b32f8b0d480547a58ba5c6fed6483a74 deleted and replaced by tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534
I20260926 19:58:46.505527 12899 tablet_service.cc:1558] Processing DeleteTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 with delete_type TABLET_DATA_DELETED (replaced by tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.509802 13013 tablet_bootstrap.cc:492] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:46.512220 12593 raft_consensus.cc:1275] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee [term 2 FOLLOWER]: Refusing update from remote peer 8112c0b43d9c4e5389aa3a18f18b58a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260926 19:58:46.517706 13426 consensus_queue.cc:1048] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:46.520571 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb593ce2cd4147febebfe442de9fdd28" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:46.521204 12594 raft_consensus.cc:2468] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:46.521993 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 with delete_type TABLET_DATA_DELETED (replaced by tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:46.523365 12695 leader_election.cc:304] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:46.548269 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_0e1b1c5a52dc4176b9b867aa57c3ff1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.552309 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e1b1c5a52dc4176b9b867aa57c3ff1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.571403 13403 ts_tablet_manager.cc:1929] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:46.577622 13092 tablet_bootstrap.cc:654] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:46.586061 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 with delete_type TABLET_DATA_DELETED (replaced by tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:46.586745 13433 raft_consensus.cc:2804] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:46.587110 13433 raft_consensus.cc:493] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:46.587463 13433 raft_consensus.cc:3060] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:46.571807 13403 log.cc:1199] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/ba3f3931a71048fd874c1131d44f7b00
I20260926 19:58:46.590070 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_fe617b43605c467aa4ded460c57b4624" } modify_external_catalogs: true
I20260926 19:58:46.590590 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_fe617b43605c467aa4ded460c57b4624" } modify_external_catalogs: true
I20260926 19:58:46.593642 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_4db2758ba1704aa58c9aeeaccfc8a825" } new_table_name: "default.Table_87fb2b97ed26421ba052e4bbb187763c" modify_external_catalogs: true
I20260926 19:58:46.596731 12387 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1e071d1293234cecb72d6a118e8d5f10" } modify_external_catalogs: true
I20260926 19:58:46.597194 12387 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1e071d1293234cecb72d6a118e8d5f10" } modify_external_catalogs: true
W20260926 19:58:46.608718 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet b32f8b0d480547a58ba5c6fed6483a74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b32f8b0d480547a58ba5c6fed6483a74 already in progress: creating tablet
I20260926 19:58:46.549103 13013 tablet_bootstrap.cc:654] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:46.517062 13401 consensus_queue.cc:1048] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
W20260926 19:58:46.630892 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a26e6b837f67445c9383599b7e52de7b has no consensus state
I20260926 19:58:46.642972 13440 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:46.643560 13425 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:46.659361 13013 tablet_bootstrap.cc:492] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:46.659796 13013 ts_tablet_manager.cc:1403] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.412s user 0.011s sys 0.002s
W20260926 19:58:46.665191 12521 consensus_peers.cc:597] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:46.654152 13403 ts_tablet_manager.cc:1950] T ba3f3931a71048fd874c1131d44f7b00 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:46.669652 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]) successfully deleted
I20260926 19:58:46.674106 13433 raft_consensus.cc:515] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.687479 13203 ts_tablet_manager.cc:1929] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:46.687469 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b2591dd15fb0435683b013e66d1ec83a" } new_table_name: "default.Table_019731a69622436792bb7a8512e0cb49" modify_external_catalogs: true
I20260926 19:58:46.695936 13424 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:46.700073 13092 tablet_bootstrap.cc:492] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:46.700673 13092 ts_tablet_manager.cc:1403] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.612s user 0.011s sys 0.000s
I20260926 19:58:46.702138 13403 tablet_replica.cc:333] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:46.703426 13403 raft_consensus.cc:2243] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:46.704252 13403 raft_consensus.cc:2272] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:46.706219 13403 ts_tablet_manager.cc:1916] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:46.711759 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_935be674b5a9419296be4ffa3464b082"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:46.712466 13203 log.cc:1199] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/ec6d07cb4c954f23be80b315a5748adc
W20260926 19:58:46.713375 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_935be674b5a9419296be4ffa3464b082 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.713568 13203 ts_tablet_manager.cc:1950] T ec6d07cb4c954f23be80b315a5748adc P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:46.714618 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 137) took 2670 ms. Trace:
I20260926 19:58:46.724608 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_2c9b200af3d54549b7c8184d0b0a3d44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.726263 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c9b200af3d54549b7c8184d0b0a3d44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:46.731863 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 45793.3us
I20260926 19:58:46.692005 13013 raft_consensus.cc:359] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.737741 13013 raft_consensus.cc:385] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:46.738183 13013 raft_consensus.cc:740] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:46.739145 13013 consensus_queue.cc:260] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.744635 13433 leader_election.cc:290] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:46.758876 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_070eaadfa595480ca36eede1de4c5a1f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.760840 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_070eaadfa595480ca36eede1de4c5a1f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.715046 13203 rpcz_store.cc:276] 0926 19:58:44.043936 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:44.075860 (+ 31924us) service_pool.cc:224] Handling call
0926 19:58:46.053199 (+1977339us) ts_tablet_manager.cc:1151] Deleting tablet ec6d07cb4c954f23be80b315a5748adc
0926 19:58:46.053238 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:46.506769 (+453531us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:46.687351 (+180582us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:46.714541 (+ 27190us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1976786}
I20260926 19:58:46.775981 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 139) took 2630 ms. Trace:
I20260926 19:58:46.776571 13203 rpcz_store.cc:276] 0926 19:58:44.146019 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:44.146164 (+ 145us) service_pool.cc:224] Handling call
0926 19:58:46.775469 (+2629305us) ts_tablet_manager.cc:1151] Deleting tablet 16897e7489e9445ba5389a4f3507a739
0926 19:58:46.775550 (+ 81us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:46.775906 (+ 356us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2628720}
I20260926 19:58:46.778317 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet ec6d07cb4c954f23be80b315a5748adc (table default.Table_56393c731f9e44fe823755a7e91ccc3e [id=137dcda542e04a92938b739a96b65ea9]) successfully deleted
I20260926 19:58:46.778620 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb593ce2cd4147febebfe442de9fdd28" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
W20260926 19:58:46.779178 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 16897e7489e9445ba5389a4f3507a739 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16897e7489e9445ba5389a4f3507a739 already in progress: creating tablet
I20260926 19:58:46.779727 12593 raft_consensus.cc:3060] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:46.711409 13092 raft_consensus.cc:359] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.789847 13092 raft_consensus.cc:385] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:46.790352 13092 raft_consensus.cc:740] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:46.790940 12391 master_service.cc:960] ReplaceTablet: received request to replace tablet 8f59be98ceef4effa15baf487c2d1140 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:46.791482 13092 consensus_queue.cc:260] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.794646 13092 ts_tablet_manager.cc:1434] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.094s user 0.008s sys 0.002s
I20260926 19:58:46.801532 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet b82d75be43e841e7abcc9e9db800aa2d
I20260926 19:58:46.803222 13013 ts_tablet_manager.cc:1434] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.143s user 0.005s sys 0.004s
I20260926 19:58:46.805821 12899 tablet_service.cc:1558] Processing DeleteTablet for tablet b82d75be43e841e7abcc9e9db800aa2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:46.810181 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet b82d75be43e841e7abcc9e9db800aa2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:46.811262 13400 tablet_replica.cc:333] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:46.812276 13400 raft_consensus.cc:2243] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:46.812812 13400 raft_consensus.cc:2272] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:46.815136 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet b82d75be43e841e7abcc9e9db800aa2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:46.821982 13400 ts_tablet_manager.cc:1916] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:46.830214 13409 raft_consensus.cc:493] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:46.830695 13409 raft_consensus.cc:515] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.833552 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_df083bf280b942dfab05341b05afadc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:46.836823 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df083bf280b942dfab05341b05afadc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:46.843672 13409 leader_election.cc:290] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:46.846423 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb593ce2cd4147febebfe442de9fdd28" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:46.848074 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd184e6295d47aeba809be6aab9ce1c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:46.848990 12595 raft_consensus.cc:2468] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:46.850168 12695 leader_election.cc:304] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:46.850955 13409 raft_consensus.cc:2804] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:46.851279 13409 raft_consensus.cc:493] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:46.851589 13409 raft_consensus.cc:3060] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:46.858618 13409 raft_consensus.cc:515] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.861316 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd184e6295d47aeba809be6aab9ce1c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:46.861835 12595 raft_consensus.cc:3060] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:46.864681 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd184e6295d47aeba809be6aab9ce1c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:46.865747 12921 raft_consensus.cc:2468] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:46.868165 12595 raft_consensus.cc:2468] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:46.869258 12695 leader_election.cc:304] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:46.870029 13432 raft_consensus.cc:2804] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:46.870467 13432 raft_consensus.cc:697] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:46.871237 13432 consensus_queue.cc:237] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.873588 13409 leader_election.cc:290] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:46.874092 13012 tablet_bootstrap.cc:700] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent opening tablet: real 0.126s user 0.002s sys 0.001s
I20260926 19:58:46.876715 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd184e6295d47aeba809be6aab9ce1c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:46.877594 12921 raft_consensus.cc:3060] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:46.882668 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 147) took 2274 ms. Trace:
I20260926 19:58:46.882781 13012 tablet_bootstrap.cc:654] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:46.882998 13203 rpcz_store.cc:276] 0926 19:58:44.608147 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:44.610242 (+ 2095us) service_pool.cc:224] Handling call
0926 19:58:46.882340 (+2272098us) ts_tablet_manager.cc:1151] Deleting tablet 06c5ef9e1bca427fb889cbb17f77f999
0926 19:58:46.882386 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:46.882615 (+ 229us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2271536}
I20260926 19:58:46.883962 13203 tablet_replica.cc:333] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:46.884658 13203 raft_consensus.cc:2243] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
W20260926 19:58:46.884580 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 06c5ef9e1bca427fb889cbb17f77f999 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06c5ef9e1bca427fb889cbb17f77f999 already in progress: creating tablet
I20260926 19:58:46.885073 13203 raft_consensus.cc:2272] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:46.885993 13013 tablet_bootstrap.cc:492] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:46.891489 13013 tablet_bootstrap.cc:654] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:46.892848 13092 tablet_bootstrap.cc:492] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
W20260926 19:58:46.901530 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13bb4e7eeef145fc85a5c412add99ce7 has no consensus state
W20260926 19:58:46.912355 12519 consensus_peers.cc:597] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:46.915158 13203 ts_tablet_manager.cc:1916] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:46.919660 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9f3804455bdf4e8f81dcbf449bdb9c03" } new_table_name: "default.Table_a8870fb0212e43b788b3fced198aed4e" modify_external_catalogs: true
I20260926 19:58:46.940436 12593 raft_consensus.cc:2468] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
W20260926 19:58:46.926016 12696 leader_election.cc:343] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:46.951757 12695 leader_election.cc:304] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:46.952711 13432 raft_consensus.cc:2804] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:46.953171 13432 raft_consensus.cc:697] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:46.953986 13432 consensus_queue.cc:237] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:46.964673 13400 ts_tablet_manager.cc:1929] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:46.965248 13400 log.cc:1199] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/b82d75be43e841e7abcc9e9db800aa2d
I20260926 19:58:46.989116 13403 ts_tablet_manager.cc:1929] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:46.994226 13203 ts_tablet_manager.cc:1929] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:46.991569 13403 log.cc:1199] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/0ea151df659944f9a094fe9410c3f4bb
I20260926 19:58:46.995574 13403 ts_tablet_manager.cc:1950] T 0ea151df659944f9a094fe9410c3f4bb P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:47.000903 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 0ea151df659944f9a094fe9410c3f4bb (table default.Table_36db24f73aca410799bcb881e2bd1bf9 [id=cb9a3efb81484b919064086496cd2d4c]) successfully deleted
I20260926 19:58:47.003521 12921 raft_consensus.cc:2468] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:47.005743 13203 log.cc:1199] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/27ccbbe1a5784e97827a31e9febd8c20
I20260926 19:58:47.019979 13400 ts_tablet_manager.cc:1950] T b82d75be43e841e7abcc9e9db800aa2d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:47.023319 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet b82d75be43e841e7abcc9e9db800aa2d (table default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa]) successfully deleted
I20260926 19:58:47.026834 13403 tablet_replica.cc:333] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:47.027124 13012 tablet_bootstrap.cc:492] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:47.027506 13012 ts_tablet_manager.cc:1403] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.538s user 0.006s sys 0.005s
I20260926 19:58:47.027670 13403 raft_consensus.cc:2243] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:47.028107 13403 raft_consensus.cc:2272] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:47.029868 13403 ts_tablet_manager.cc:1916] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:47.030078 13012 raft_consensus.cc:359] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.030655 13012 raft_consensus.cc:385] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.030896 13012 raft_consensus.cc:740] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.031607 13012 consensus_queue.cc:260] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.034123 13012 ts_tablet_manager.cc:1434] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260926 19:58:47.006904 13203 ts_tablet_manager.cc:1950] T 27ccbbe1a5784e97827a31e9febd8c20 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:47.043206 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 149) took 2193 ms. Trace:
I20260926 19:58:47.043561 13203 rpcz_store.cc:276] 0926 19:58:44.849683 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:44.849773 (+ 90us) service_pool.cc:224] Handling call
0926 19:58:46.883825 (+2034052us) ts_tablet_manager.cc:1151] Deleting tablet 27ccbbe1a5784e97827a31e9febd8c20
0926 19:58:46.883863 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:46.970619 (+ 86756us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:46.994101 (+ 23482us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:47.043142 (+ 49041us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2033564}
I20260926 19:58:47.052709 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 54553d20d48148ae88fa4cc2ce021779 (DEFAULT_TABLE table=default.Table_7a4372f262834b68bade3baafd4658b7 [id=ec59131fc8ed4403a5decd1901f6d962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.054175 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54553d20d48148ae88fa4cc2ce021779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.059495 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 27ccbbe1a5784e97827a31e9febd8c20 (table default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1 [id=36e060a7ae90436d8c3917819816609c]) successfully deleted
I20260926 19:58:47.062613 13012 tablet_bootstrap.cc:492] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:47.067062 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 158) took 1391 ms. Trace:
I20260926 19:58:47.069844 13203 rpcz_store.cc:276] 0926 19:58:45.675982 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:45.676107 (+ 125us) service_pool.cc:224] Handling call
0926 19:58:47.066554 (+1390447us) ts_tablet_manager.cc:1151] Deleting tablet 700d05149a934d5e8bec9cc41873ad3e
0926 19:58:47.066620 (+ 66us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:47.066983 (+ 363us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1389339}
I20260926 19:58:47.075995 12389 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
I20260926 19:58:47.076787 12389 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_13a9930424e74b249d88b4f42195f0ad [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 2 got 0
W20260926 19:58:47.078451 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 700d05149a934d5e8bec9cc41873ad3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 700d05149a934d5e8bec9cc41873ad3e already in progress: creating tablet
I20260926 19:58:47.083021 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 159) took 1396 ms. Trace:
I20260926 19:58:47.083492 13203 rpcz_store.cc:276] 0926 19:58:45.687015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:45.687365 (+ 350us) service_pool.cc:224] Handling call
0926 19:58:47.082575 (+1395210us) ts_tablet_manager.cc:1151] Deleting tablet 89c8cbb097a644218e0ca033a6fdb4a2
0926 19:58:47.082624 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:47.082938 (+ 314us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1394094}
W20260926 19:58:47.085260 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 89c8cbb097a644218e0ca033a6fdb4a2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89c8cbb097a644218e0ca033a6fdb4a2 already in progress: creating tablet
I20260926 19:58:47.091620 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 160) took 1279 ms. Trace:
I20260926 19:58:47.091939 13203 rpcz_store.cc:276] 0926 19:58:45.812379 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:45.812481 (+ 102us) service_pool.cc:224] Handling call
0926 19:58:47.091262 (+1278781us) ts_tablet_manager.cc:1151] Deleting tablet acdb4bd8cd58444cb1d6d4b5610de20d
0926 19:58:47.091306 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:47.091565 (+ 259us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1278220}
W20260926 19:58:47.093147 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet acdb4bd8cd58444cb1d6d4b5610de20d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet acdb4bd8cd58444cb1d6d4b5610de20d already in progress: creating tablet
I20260926 19:58:47.096275 13012 tablet_bootstrap.cc:654] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.099061 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 54553d20d48148ae88fa4cc2ce021779 (DEFAULT_TABLE table=default.Table_7a4372f262834b68bade3baafd4658b7 [id=ec59131fc8ed4403a5decd1901f6d962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.100586 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54553d20d48148ae88fa4cc2ce021779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.101920 13432 raft_consensus.cc:493] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:47.102089 12389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 45) took 828 ms (client timeout 1000 ms). Trace:
I20260926 19:58:47.102341 12390 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_13a9930424e74b249d88b4f42195f0ad [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 2 got 0
I20260926 19:58:47.102377 13432 raft_consensus.cc:515] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:47.102537 12389 rpcz_store.cc:269] 0926 19:58:46.273305 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.274105 (+ 800us) service_pool.cc:224] Handling call
0926 19:58:47.102027 (+827922us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":245100,"reported_tablets":2}
I20260926 19:58:47.103937 12389 master_service.cc:960] ReplaceTablet: received request to replace tablet 16df0828847d4e7db5702d84082fa402 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:47.105792 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 54553d20d48148ae88fa4cc2ce021779 (DEFAULT_TABLE table=default.Table_7a4372f262834b68bade3baafd4658b7 [id=ec59131fc8ed4403a5decd1901f6d962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.106868 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54553d20d48148ae88fa4cc2ce021779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.107183 12388 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_13a9930424e74b249d88b4f42195f0ad [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 2 got 0
I20260926 19:58:47.107805 12388 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
I20260926 19:58:47.108297 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 14ca451e5139456b9deb41a4fc635c87 (DEFAULT_TABLE table=default.Table_49d5eefaa2aa4282b699143ac47a444d [id=407f8ec0fc844a518e38a1abb7d524fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.109288 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ca451e5139456b9deb41a4fc635c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.110939 13401 raft_consensus.cc:493] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:47.111364 13401 raft_consensus.cc:515] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.114015 13401 leader_election.cc:290] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:47.119840 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 14ca451e5139456b9deb41a4fc635c87 (DEFAULT_TABLE table=default.Table_49d5eefaa2aa4282b699143ac47a444d [id=407f8ec0fc844a518e38a1abb7d524fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.121125 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ca451e5139456b9deb41a4fc635c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.138151 13203 tablet_replica.cc:333] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:47.139418 13203 raft_consensus.cc:2243] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:47.139959 13203 raft_consensus.cc:2272] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:47.145490 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3215bbd5be56438491a73925ac9ccbe6" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:47.158527 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_e0ad88ad494c42759968cb1dc8501282" } modify_external_catalogs: true
I20260926 19:58:47.159227 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_e0ad88ad494c42759968cb1dc8501282" } modify_external_catalogs: true
I20260926 19:58:47.162240 13432 leader_election.cc:290] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:47.163302 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 14ca451e5139456b9deb41a4fc635c87 (DEFAULT_TABLE table=default.Table_49d5eefaa2aa4282b699143ac47a444d [id=407f8ec0fc844a518e38a1abb7d524fe]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:47.163497 12519 leader_election.cc:343] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 3215bbd5be56438491a73925ac9ccbe6
I20260926 19:58:47.164340 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ca451e5139456b9deb41a4fc635c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.166075 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 3066a65ce44b4387a6f96124d9f476fb (DEFAULT_TABLE table=default.Table_f07ac4e733db421bb4a3efc6f937c310 [id=c2dfbc1199344fc187669bf813fcc16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.166953 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3066a65ce44b4387a6f96124d9f476fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.170817 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189eaca5fb2b4b0d83a2873e0495e5ff" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:47.171377 12593 raft_consensus.cc:2468] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:47.172588 12695 leader_election.cc:304] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:47.174193 12921 raft_consensus.cc:2468] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:47.175953 12924 raft_consensus.cc:3060] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:47.175856 12521 leader_election.cc:304] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:47.177285 13401 raft_consensus.cc:2804] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:47.177739 13401 raft_consensus.cc:493] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:47.178114 13401 raft_consensus.cc:3060] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:47.191115 13401 raft_consensus.cc:515] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.194389 13013 tablet_bootstrap.cc:492] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:47.194376 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189eaca5fb2b4b0d83a2873e0495e5ff" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:47.194784 13013 ts_tablet_manager.cc:1403] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.309s user 0.037s sys 0.035s
W20260926 19:58:47.196485 12696 leader_election.cc:343] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:47.186555 13432 raft_consensus.cc:2804] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:47.204970 13203 ts_tablet_manager.cc:1916] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:47.203486 13432 raft_consensus.cc:493] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:47.207677 13432 raft_consensus.cc:3060] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:47.214536 13013 raft_consensus.cc:359] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:47.215197 13013 raft_consensus.cc:385] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.215443 13013 raft_consensus.cc:740] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.216121 13013 consensus_queue.cc:260] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:47.218812 13013 ts_tablet_manager.cc:1434] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.024s user 0.006s sys 0.001s
I20260926 19:58:47.221285 13013 tablet_bootstrap.cc:492] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:47.241720 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 3066a65ce44b4387a6f96124d9f476fb (DEFAULT_TABLE table=default.Table_f07ac4e733db421bb4a3efc6f937c310 [id=c2dfbc1199344fc187669bf813fcc16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.242609 13013 tablet_bootstrap.cc:654] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.243059 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3066a65ce44b4387a6f96124d9f476fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.264330 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 3066a65ce44b4387a6f96124d9f476fb (DEFAULT_TABLE table=default.Table_f07ac4e733db421bb4a3efc6f937c310 [id=c2dfbc1199344fc187669bf813fcc16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.265297 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3066a65ce44b4387a6f96124d9f476fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.266696 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 74fe405570f8474fb7a1f841a783a90a (DEFAULT_TABLE table=default.Table_acdb314f8df24fea96383e9e16a5efe6 [id=2d2ae06595164e708751168edcdad523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.267622 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fe405570f8474fb7a1f841a783a90a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.270160 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 74fe405570f8474fb7a1f841a783a90a (DEFAULT_TABLE table=default.Table_acdb314f8df24fea96383e9e16a5efe6 [id=2d2ae06595164e708751168edcdad523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.271071 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fe405570f8474fb7a1f841a783a90a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.233219 13432 raft_consensus.cc:515] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.271692 13013 tablet_bootstrap.cc:492] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:47.272071 13013 ts_tablet_manager.cc:1403] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.000s
I20260926 19:58:47.273216 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189eaca5fb2b4b0d83a2873e0495e5ff" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:47.273826 12593 raft_consensus.cc:3060] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:47.274994 13013 raft_consensus.cc:359] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.275586 13013 raft_consensus.cc:385] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.275818 13013 raft_consensus.cc:740] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.285429 13432 leader_election.cc:290] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:47.281350 12593 raft_consensus.cc:2468] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:47.291293 12695 leader_election.cc:304] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:47.289783 13013 consensus_queue.cc:260] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.295001 13452 tablet.cc:1751] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:47.297132 13013 ts_tablet_manager.cc:1434] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.025s user 0.003s sys 0.005s
I20260926 19:58:47.295360 13092 tablet_bootstrap.cc:654] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.300480 13409 raft_consensus.cc:2804] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:47.300915 13409 raft_consensus.cc:697] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:47.301755 13409 consensus_queue.cc:237] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.313225 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189eaca5fb2b4b0d83a2873e0495e5ff" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:47.318660 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
I20260926 19:58:47.320914 13092 tablet_bootstrap.cc:492] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:47.321314 13092 ts_tablet_manager.cc:1403] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.429s user 0.012s sys 0.001s
I20260926 19:58:47.323565 13092 raft_consensus.cc:359] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.324147 13092 raft_consensus.cc:385] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.324389 13092 raft_consensus.cc:740] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.330391 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3215bbd5be56438491a73925ac9ccbe6" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:47.331177 12921 raft_consensus.cc:3060] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:47.332142 12393 catalog_manager.cc:5838] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:47.337734 13092 consensus_queue.cc:260] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:47.340131 12519 leader_election.cc:343] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 3215bbd5be56438491a73925ac9ccbe6
W20260926 19:58:47.343771 12696 leader_election.cc:343] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:47.346625 13013 tablet_bootstrap.cc:492] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:47.347977 13092 ts_tablet_manager.cc:1434] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.026s user 0.004s sys 0.001s
I20260926 19:58:47.352810 13013 tablet_bootstrap.cc:654] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.366020 13092 tablet_bootstrap.cc:492] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:47.377638 12739 tablet_service.cc:1511] Processing CreateTablet for tablet fe567f5d4c8c41ff9d681ca9ad3970e8 (DEFAULT_TABLE table=default.Table_0718f84db29d46d08463ca1b53df6b71 [id=f8c513b6b1c64309be6eb14fb7ae2215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.378607 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe567f5d4c8c41ff9d681ca9ad3970e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.380223 12899 tablet_service.cc:1511] Processing CreateTablet for tablet fe567f5d4c8c41ff9d681ca9ad3970e8 (DEFAULT_TABLE table=default.Table_0718f84db29d46d08463ca1b53df6b71 [id=f8c513b6b1c64309be6eb14fb7ae2215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.381136 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe567f5d4c8c41ff9d681ca9ad3970e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.391124 13092 tablet_bootstrap.cc:654] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.393710 13012 tablet_bootstrap.cc:492] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:47.394124 13012 ts_tablet_manager.cc:1403] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.332s user 0.009s sys 0.006s
I20260926 19:58:47.397219 13012 raft_consensus.cc:359] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.397886 13012 raft_consensus.cc:385] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.398142 13012 raft_consensus.cc:740] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.398852 13012 consensus_queue.cc:260] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.408291 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]). Expected version 2 got 0
I20260926 19:58:47.414934 13403 ts_tablet_manager.cc:1929] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:47.415395 13403 log.cc:1199] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/e550463602ff4b5f98e10dd593897bcf
I20260926 19:58:47.416150 13403 ts_tablet_manager.cc:1950] T e550463602ff4b5f98e10dd593897bcf P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:47.416180 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b68b74a1774041749a10f4006f15ece1" } new_table_name: "default.Table_f136506f88074adcb77c8ab2bc88ce51" modify_external_catalogs: true
I20260926 19:58:47.418902 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 154) took 1266 ms. Trace:
I20260926 19:58:47.435951 13012 ts_tablet_manager.cc:1434] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20260926 19:58:47.371986 13401 leader_election.cc:290] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:47.454384 13012 tablet_bootstrap.cc:492] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:47.451552 13092 tablet_bootstrap.cc:492] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:47.457816 13092 ts_tablet_manager.cc:1403] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.092s user 0.008s sys 0.006s
I20260926 19:58:47.457618 13403 rpcz_store.cc:276] 0926 19:58:46.152446 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.152562 (+ 116us) service_pool.cc:224] Handling call
0926 19:58:47.026663 (+874101us) ts_tablet_manager.cc:1151] Deleting tablet e550463602ff4b5f98e10dd593897bcf
0926 19:58:47.026708 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:47.144602 (+117894us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:47.414800 (+270198us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:47.418810 (+ 4010us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":873614}
I20260926 19:58:47.459465 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet e550463602ff4b5f98e10dd593897bcf (table default.Table_1e071d1293234cecb72d6a118e8d5f10 [id=21d0c2efe78e438e8fad11d811c463ed]) successfully deleted
I20260926 19:58:47.460237 13092 raft_consensus.cc:359] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.460582 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 156) took 1221 ms. Trace:
I20260926 19:58:47.460835 13092 raft_consensus.cc:385] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.461099 13092 raft_consensus.cc:740] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.409231 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:47.465739 13092 consensus_queue.cc:260] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.474741 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9d1e6cd14bd8461faff475b85d9b9368"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:47.476788 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d1e6cd14bd8461faff475b85d9b9368 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:47.481009 13092 ts_tablet_manager.cc:1434] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20260926 19:58:47.487074 12733 tablet_service.cc:1511] Processing CreateTablet for tablet a9bcec32d2504a4188dd0c6c3486f7ff (DEFAULT_TABLE table=default.Table_27b2665fcc774f07b2aaee9205ea4ad4 [id=45ea052fc8144e5c8357d55422baa580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.488778 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bcec32d2504a4188dd0c6c3486f7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.491732 12904 tablet_service.cc:1511] Processing CreateTablet for tablet a9bcec32d2504a4188dd0c6c3486f7ff (DEFAULT_TABLE table=default.Table_27b2665fcc774f07b2aaee9205ea4ad4 [id=45ea052fc8144e5c8357d55422baa580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.461094 13403 rpcz_store.cc:276] 0926 19:58:46.238873 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.239443 (+ 570us) service_pool.cc:224] Handling call
0926 19:58:47.460207 (+1220764us) ts_tablet_manager.cc:1151] Deleting tablet cd86f01d82d041f4852f83f2547e697f
0926 19:58:47.460257 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:47.460531 (+ 274us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1220072}
I20260926 19:58:47.493364 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bcec32d2504a4188dd0c6c3486f7ff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:47.493764 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet cd86f01d82d041f4852f83f2547e697f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd86f01d82d041f4852f83f2547e697f already in progress: creating tablet
I20260926 19:58:47.494742 13403 tablet_replica.cc:333] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:47.495841 13092 tablet_bootstrap.cc:492] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:47.496043 13013 tablet_bootstrap.cc:492] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:47.496431 13013 ts_tablet_manager.cc:1403] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.150s user 0.007s sys 0.002s
I20260926 19:58:47.497344 13409 raft_consensus.cc:493] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:47.497963 13409 raft_consensus.cc:515] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.498878 13013 raft_consensus.cc:359] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.499567 13013 raft_consensus.cc:385] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.499810 13013 raft_consensus.cc:740] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.500464 13409 leader_election.cc:290] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:47.502417 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1455b99a65ff44ff933825196708f485" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:47.504591 12696 leader_election.cc:343] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:47.505976 13012 tablet_bootstrap.cc:654] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.500385 13013 consensus_queue.cc:260] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.503051 13203 ts_tablet_manager.cc:1929] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:47.514456 13203 log.cc:1199] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/cd92a1a62ab24647bef7c9eefdf9c9d8
I20260926 19:58:47.515157 13203 ts_tablet_manager.cc:1950] T cd92a1a62ab24647bef7c9eefdf9c9d8 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:47.515527 13092 tablet_bootstrap.cc:654] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.517946 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 162) took 1638 ms. Trace:
I20260926 19:58:47.518338 13203 rpcz_store.cc:276] 0926 19:58:45.879434 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.011903 (+132469us) service_pool.cc:224] Handling call
0926 19:58:47.137904 (+1126001us) ts_tablet_manager.cc:1151] Deleting tablet cd92a1a62ab24647bef7c9eefdf9c9d8
0926 19:58:47.137978 (+ 74us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:47.422552 (+284574us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:47.502944 (+ 80392us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:47.517893 (+ 14949us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1125192}
I20260926 19:58:47.519418 13203 tablet_replica.cc:333] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:47.520213 13203 raft_consensus.cc:2243] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:47.521486 13013 ts_tablet_manager.cc:1434] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260926 19:58:47.521823 13203 raft_consensus.cc:2272] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:47.526068 13203 ts_tablet_manager.cc:1916] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:47.527237 13013 tablet_bootstrap.cc:492] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:47.539167 13012 tablet_bootstrap.cc:492] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:47.539582 13012 ts_tablet_manager.cc:1403] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.085s user 0.007s sys 0.002s
I20260926 19:58:47.540370 13092 tablet_bootstrap.cc:492] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:47.540736 13092 ts_tablet_manager.cc:1403] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.001s
I20260926 19:58:47.543090 13092 raft_consensus.cc:359] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.543668 13092 raft_consensus.cc:385] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.543941 13092 raft_consensus.cc:740] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.544153 12921 raft_consensus.cc:2468] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:47.544782 13403 raft_consensus.cc:2243] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:47.546150 13012 raft_consensus.cc:359] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:47.547106 13403 raft_consensus.cc:2272] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:47.547066 13012 raft_consensus.cc:385] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.547400 13012 raft_consensus.cc:740] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.548403 13092 consensus_queue.cc:260] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.550055 13403 ts_tablet_manager.cc:1916] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:47.552424 12382 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 13bb4e7eeef145fc85a5c412add99ce7 (table default.Table_019731a69622436792bb7a8512e0cb49 [id=f2cc65dc707940e5a763c0d4b768a072], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 13bb4e7eeef145fc85a5c412add99ce7 has no consensus state
I20260926 19:58:47.558256 13012 consensus_queue.cc:260] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:47.562419 13092 ts_tablet_manager.cc:1434] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260926 19:58:47.563323 13092 tablet_bootstrap.cc:492] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:47.567131 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 7645307e0999482bbe3329c8d61cc567 (DEFAULT_TABLE table=default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.568619 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7645307e0999482bbe3329c8d61cc567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.582881 13012 ts_tablet_manager.cc:1434] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.043s user 0.006s sys 0.001s
I20260926 19:58:47.594190 13012 tablet_bootstrap.cc:492] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:47.596966 13092 tablet_bootstrap.cc:654] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.610265 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ae3912e6fea44f88a074a4bbed57b0cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:47.611368 13426 raft_consensus.cc:493] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:47.611877 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae3912e6fea44f88a074a4bbed57b0cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:47.611840 13426 raft_consensus.cc:515] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.614343 13426 leader_election.cc:290] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:47.614516 13409 raft_consensus.cc:493] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:47.614941 13409 raft_consensus.cc:515] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:47.616920 13409 leader_election.cc:290] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:47.619529 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:47.621078 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7d5e580f5645ab95fa03d5f669dc32" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:47.622454 12696 leader_election.cc:343] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:47.622326 12857 leader_election.cc:343] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:47.623872 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet cd92a1a62ab24647bef7c9eefdf9c9d8 (table default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]) successfully deleted
I20260926 19:58:47.626098 13012 tablet_bootstrap.cc:654] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.635339 12521 leader_election.cc:304] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:47.651839 13012 tablet_bootstrap.cc:492] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:47.652377 13012 ts_tablet_manager.cc:1403] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.002s
I20260926 19:58:47.654634 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 7645307e0999482bbe3329c8d61cc567 (DEFAULT_TABLE table=default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.655635 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7645307e0999482bbe3329c8d61cc567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.656364 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a14b58f3fb9842d6a5ab2711958ddec9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:47.379376 12561 tablet_service.cc:1511] Processing CreateTablet for tablet fe567f5d4c8c41ff9d681ca9ad3970e8 (DEFAULT_TABLE table=default.Table_0718f84db29d46d08463ca1b53df6b71 [id=f8c513b6b1c64309be6eb14fb7ae2215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.659566 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe567f5d4c8c41ff9d681ca9ad3970e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.659751 12559 tablet_service.cc:1511] Processing CreateTablet for tablet a9bcec32d2504a4188dd0c6c3486f7ff (DEFAULT_TABLE table=default.Table_27b2665fcc774f07b2aaee9205ea4ad4 [id=45ea052fc8144e5c8357d55422baa580]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:47.663337 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a14b58f3fb9842d6a5ab2711958ddec9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:47.664780 13012 raft_consensus.cc:359] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.665681 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9bcec32d2504a4188dd0c6c3486f7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.678473 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 7645307e0999482bbe3329c8d61cc567 (DEFAULT_TABLE table=default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.679869 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7645307e0999482bbe3329c8d61cc567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.693651 13012 raft_consensus.cc:385] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.694128 13012 raft_consensus.cc:740] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.695228 13012 consensus_queue.cc:260] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.375938 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 74fe405570f8474fb7a1f841a783a90a (DEFAULT_TABLE table=default.Table_acdb314f8df24fea96383e9e16a5efe6 [id=2d2ae06595164e708751168edcdad523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.700855 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fe405570f8474fb7a1f841a783a90a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:47.702104 12568 consensus_meta.cc:287] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee: Time spent flushing consensus metadata: real 0.550s user 0.010s sys 0.000s
I20260926 19:58:47.705619 12591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7d5e580f5645ab95fa03d5f669dc32" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:47.706490 12591 raft_consensus.cc:2468] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:47.708494 12695 leader_election.cc:304] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:47.708839 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:47.709452 12593 raft_consensus.cc:2468] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:47.709631 13432 raft_consensus.cc:2804] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:47.710119 13432 raft_consensus.cc:493] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:47.710574 13432 raft_consensus.cc:3060] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:47.710696 12858 leader_election.cc:304] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
W20260926 19:58:47.711687 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 76975.7us
I20260926 19:58:47.712378 13426 raft_consensus.cc:2804] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:47.712844 13426 raft_consensus.cc:493] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:47.713212 13426 raft_consensus.cc:3060] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:47.714476 13012 ts_tablet_manager.cc:1434] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.062s user 0.005s sys 0.000s
I20260926 19:58:47.716166 13012 tablet_bootstrap.cc:492] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:47.719939 13426 raft_consensus.cc:515] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.727255 13384 raft_consensus.cc:493] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:47.727707 13384 raft_consensus.cc:515] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.729631 13384 leader_election.cc:290] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:47.731926 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb593ce2cd4147febebfe442de9fdd28" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:47.732172 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb593ce2cd4147febebfe442de9fdd28" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:47.732983 12593 raft_consensus.cc:2393] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:47.734694 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 8a0e37779b0e415a930c8d3b5317bcd0 (DEFAULT_TABLE table=default.Table_0911867d73d545f7bf86c97f052dee59 [id=365e3b791c3742cab761c833f8e2123f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.734862 12858 leader_election.cc:304] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:47.735625 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a0e37779b0e415a930c8d3b5317bcd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.738747 13384 raft_consensus.cc:3060] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:47.740187 12695 leader_election.cc:343] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 1455b99a65ff44ff933825196708f485
I20260926 19:58:47.740653 12695 leader_election.cc:304] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:47.741343 13409 raft_consensus.cc:2749] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:47.745296 13384 raft_consensus.cc:2749] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:47.746474 13461 raft_consensus.cc:493] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:47.747710 13012 tablet_bootstrap.cc:654] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.747210 13461 raft_consensus.cc:515] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.753024 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 8a0e37779b0e415a930c8d3b5317bcd0 (DEFAULT_TABLE table=default.Table_0911867d73d545f7bf86c97f052dee59 [id=365e3b791c3742cab761c833f8e2123f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.757086 12740 tablet_service.cc:1511] Processing CreateTablet for tablet de85e3e17bf340f8895aff3f5421c85c (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.758051 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de85e3e17bf340f8895aff3f5421c85c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.758044 12900 tablet_service.cc:1511] Processing CreateTablet for tablet de85e3e17bf340f8895aff3f5421c85c (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.720992 13432 raft_consensus.cc:515] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:47.769258 13401 raft_consensus.cc:2804] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:47.771816 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:47.771900 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7d5e580f5645ab95fa03d5f669dc32" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:47.774693 12696 leader_election.cc:343] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:47.774869 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de85e3e17bf340f8895aff3f5421c85c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.775522 13409 consensus_queue.cc:1048] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:47.782614 13432 leader_election.cc:290] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:47.787567 13426 leader_election.cc:290] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:47.788134 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:47.790158 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a0e37779b0e415a930c8d3b5317bcd0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:47.790350 12857 leader_election.cc:343] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:47.815342 13092 tablet_bootstrap.cc:492] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:47.815762 13092 ts_tablet_manager.cc:1403] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.253s user 0.008s sys 0.006s
I20260926 19:58:47.781322 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7d5e580f5645ab95fa03d5f669dc32" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:47.823014 12595 raft_consensus.cc:3060] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:47.829393 13092 raft_consensus.cc:359] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.830031 13092 raft_consensus.cc:385] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.830268 13092 raft_consensus.cc:740] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.834466 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 36c14fcb0577402abe5fbb2645bd1856 (DEFAULT_TABLE table=default.Table_9ad397471def442f87f3594e163fe1fd [id=2b7641258c3f4856ad819887d6cf3bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.835799 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36c14fcb0577402abe5fbb2645bd1856. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:47.829056 12521 leader_election.cc:343] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:47.852682 13203 ts_tablet_manager.cc:1929] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:47.853266 13203 log.cc:1199] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/ba3f3931a71048fd874c1131d44f7b00
I20260926 19:58:47.855382 13092 consensus_queue.cc:260] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.857304 13092 ts_tablet_manager.cc:1434] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
I20260926 19:58:47.861264 12561 tablet_service.cc:1511] Processing CreateTablet for tablet de85e3e17bf340f8895aff3f5421c85c (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.864635 13012 tablet_bootstrap.cc:492] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:47.865053 13012 ts_tablet_manager.cc:1403] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.150s user 0.008s sys 0.001s
I20260926 19:58:47.868327 12391 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 8f59be98ceef4effa15baf487c2d1140
I20260926 19:58:47.867354 13012 raft_consensus.cc:359] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.868803 13012 raft_consensus.cc:385] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:47.869069 13012 raft_consensus.cc:740] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.869773 12391 catalog_manager.cc:6934] ReplaceTablet: tablet 8f59be98ceef4effa15baf487c2d1140 deleted and replaced by tablet 9411d3bca81641a4b3dfaffb2dc2bb51
I20260926 19:58:47.870139 12391 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:45734 (request call id 240) took 1079 ms. Trace:
I20260926 19:58:47.870957 12899 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f59be98ceef4effa15baf487c2d1140 with delete_type TABLET_DATA_DELETED (replaced by tablet 9411d3bca81641a4b3dfaffb2dc2bb51) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:47.873071 13012 consensus_queue.cc:260] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.874164 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet d48ee26c0aaa4abc8e08a1a2dadb028d
I20260926 19:58:47.875591 12387 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:45734 (request call id 235) took 1279 ms. Trace:
I20260926 19:58:47.877942 13012 ts_tablet_manager.cc:1434] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.013s user 0.004s sys 0.003s
I20260926 19:58:47.881458 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:47.875869 12387 rpcz_store.cc:276] 0926 19:58:46.596339 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.596450 (+ 111us) service_pool.cc:224] Handling call
0926 19:58:46.597679 (+ 1229us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:46.598236 (+ 557us) catalog_manager.cc:2854] Modifying in-memory table state
0926 19:58:46.598364 (+ 128us) catalog_manager.cc:2863] Locking tablets
0926 19:58:47.067127 (+468763us) catalog_manager.cc:2877] Removing table and tablets from system table
0926 19:58:47.873776 (+806649us) catalog_manager.cc:2895] Removing table from by-name map
0926 19:58:47.873970 (+ 194us) catalog_manager.cc:2911] Committing in-memory state
0926 19:58:47.874080 (+ 110us) catalog_manager.cc:2915] Aborting table tasks
0926 19:58:47.875531 (+ 1451us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:47.766726 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:47.767123 (+ 397us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:47.767149 (+ 26us) write_op.cc:435] Acquired schema lock
0926 19:58:47.767162 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:47.767456 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:47.767495 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:47.767522 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:47.767764 (+ 242us) tablet.cc:701] Row locks acquired
0926 19:58:47.767778 (+ 14us) write_op.cc:260] PREPARE: finished
0926 19:58:47.767875 (+ 97us) write_op.cc:270] Start()
0926 19:58:47.767953 (+ 78us) write_op.cc:276] Timestamp: P: 1790452727767831 usec, L: 0
0926 19:58:47.767967 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:47.768452 (+ 485us) log.cc:844] Serialized 765 byte log entry
0926 19:58:47.864437 (+ 95985us) op_driver.cc:464] REPLICATION: finished
0926 19:58:47.866763 (+ 2326us) write_op.cc:301] APPLY: starting
0926 19:58:47.866805 (+ 42us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:47.866892 (+ 87us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:47.866959 (+ 67us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:47.867177 (+ 218us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:47.867286 (+ 109us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:58:47.867323 (+ 37us) write_op.cc:312] APPLY: finished
0926 19:58:47.867460 (+ 137us) log.cc:844] Serialized 29 byte log entry
0926 19:58:47.867572 (+ 112us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:47.867646 (+ 74us) write_op.cc:454] Released schema lock
0926 19:58:47.867672 (+ 26us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2200,"apply.run_cpu_time_us":1062,"apply.run_wall_time_us":1061,"num_ops":2,"prepare.queue_time_us":697241,"prepare.run_cpu_time_us":1935,"prepare.run_wall_time_us":3295,"replication_time_us":96373,"threads_started":1}]]}
I20260926 19:58:47.884859 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f59be98ceef4effa15baf487c2d1140 with delete_type TABLET_DATA_DELETED (replaced by tablet 9411d3bca81641a4b3dfaffb2dc2bb51) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:47.885586 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:47.886282 12387 master_service.cc:960] ReplaceTablet: received request to replace tablet 59f7dc92330d400f93f427f3f79ad826 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:47.887181 13092 tablet_bootstrap.cc:492] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:47.870462 12391 rpcz_store.cc:276] 0926 19:58:46.790718 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.790813 (+ 95us) service_pool.cc:224] Handling call
0926 19:58:46.830083 (+ 39270us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cadb0e1 00007fb36a6a3d37 00007fb36a6a3cc2 00007fb36a6a3bf1 00007fb36a6a3b6c 00007fb36a6a2e22 00007fb36892d09c 00007fb36892c4d6
0926 19:58:47.870083 (+1040000us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:47.678148 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:47.678649 (+ 501us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:47.678682 (+ 33us) write_op.cc:435] Acquired schema lock
0926 19:58:47.678701 (+ 19us) tablet.cc:662] Decoding operations
0926 19:58:47.679085 (+ 384us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:47.679142 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:47.679175 (+ 33us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:47.679433 (+ 258us) tablet.cc:701] Row locks acquired
0926 19:58:47.679454 (+ 21us) write_op.cc:260] PREPARE: finished
0926 19:58:47.679594 (+ 140us) write_op.cc:270] Start()
0926 19:58:47.679699 (+ 105us) write_op.cc:276] Timestamp: P: 1790452727679540 usec, L: 0
0926 19:58:47.679722 (+ 23us) op_driver.cc:348] REPLICATION: starting
0926 19:58:47.680349 (+ 627us) log.cc:844] Serialized 631 byte log entry
0926 19:58:47.862860 (+182511us) op_driver.cc:464] REPLICATION: finished
0926 19:58:47.864878 (+ 2018us) write_op.cc:301] APPLY: starting
0926 19:58:47.864928 (+ 50us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:47.865046 (+ 118us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:47.865134 (+ 88us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:47.865471 (+ 337us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:47.865880 (+ 409us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:47.865919 (+ 39us) write_op.cc:312] APPLY: finished
0926 19:58:47.866090 (+ 171us) log.cc:844] Serialized 29 byte log entry
0926 19:58:47.866220 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:47.866281 (+ 61us) write_op.cc:454] Released schema lock
0926 19:58:47.866309 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:47.866517 (+ 208us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:47.866538 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":86520704,"child_traces":[["op",{"apply.queue_time_us":1866,"apply.run_cpu_time_us":1416,"apply.run_wall_time_us":1597,"num_ops":2,"prepare.queue_time_us":612898,"prepare.run_cpu_time_us":2591,"prepare.run_wall_time_us":88473,"raft.queue_time_us":96,"raft.run_cpu_time_us":717,"raft.run_wall_time_us":717,"replication_time_us":183024}]]}
I20260926 19:58:47.891438 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_8f5fde2cc891426eb3123a67b46f1951"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:47.893381 13409 consensus_queue.cc:1048] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:47.898679 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f5fde2cc891426eb3123a67b46f1951 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:47.899945 13012 tablet_bootstrap.cc:492] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:47.907236 13012 tablet_bootstrap.cc:654] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.910386 13092 tablet_bootstrap.cc:654] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.920130 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 36c14fcb0577402abe5fbb2645bd1856 (DEFAULT_TABLE table=default.Table_9ad397471def442f87f3594e163fe1fd [id=2b7641258c3f4856ad819887d6cf3bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:47.921257 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36c14fcb0577402abe5fbb2645bd1856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.922180 13013 tablet_bootstrap.cc:700] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee: Time spent opening tablet: real 0.113s user 0.001s sys 0.000s
I20260926 19:58:47.924297 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de85e3e17bf340f8895aff3f5421c85c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:47.931604 12565 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f59be98ceef4effa15baf487c2d1140 with delete_type TABLET_DATA_DELETED (replaced by tablet 9411d3bca81641a4b3dfaffb2dc2bb51) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:47.932272 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:47.933066 13409 consensus_queue.cc:1048] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:47.936525 13409 consensus_queue.cc:1048] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:47.938082 13013 tablet_bootstrap.cc:654] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:47.946117 13384 raft_consensus.cc:493] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:47.946599 13384 raft_consensus.cc:515] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.949014 13384 leader_election.cc:290] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:47.950898 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 8a0e37779b0e415a930c8d3b5317bcd0 (DEFAULT_TABLE table=default.Table_0911867d73d545f7bf86c97f052dee59 [id=365e3b791c3742cab761c833f8e2123f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:47.958405 12857 leader_election.cc:343] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: 27ccbbe1a5784e97827a31e9febd8c20
I20260926 19:58:47.988135 13468 raft_consensus.cc:493] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:47.988428 13012 tablet_bootstrap.cc:492] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:47.988808 13012 ts_tablet_manager.cc:1403] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.089s user 0.020s sys 0.003s
I20260926 19:58:47.988591 13468 raft_consensus.cc:515] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:47.990825 13468 leader_election.cc:290] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:47.992409 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:47.994817 12696 leader_election.cc:343] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:47.997169 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:47.997851 12757 raft_consensus.cc:2468] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:47.953290 12595 raft_consensus.cc:2468] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:47.975315 13203 ts_tablet_manager.cc:1950] T ba3f3931a71048fd874c1131d44f7b00 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:48.003578 13092 tablet_bootstrap.cc:492] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:48.003980 13092 ts_tablet_manager.cc:1403] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.117s user 0.008s sys 0.001s
I20260926 19:58:48.006264 13092 raft_consensus.cc:359] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.006855 13092 raft_consensus.cc:385] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:48.007089 13092 raft_consensus.cc:740] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:48.009439 13012 raft_consensus.cc:359] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.010074 13012 raft_consensus.cc:385] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:48.010300 13012 raft_consensus.cc:740] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:47.992925 13461 leader_election.cc:290] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:47.994417 12591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:47.997952 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:48.004366 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a0e37779b0e415a930c8d3b5317bcd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:48.021864 13092 consensus_queue.cc:260] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.020512 13012 consensus_queue.cc:260] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.032285 13012 ts_tablet_manager.cc:1434] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.043s user 0.004s sys 0.004s
I20260926 19:58:48.034586 13012 tablet_bootstrap.cc:492] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:48.041447 13092 ts_tablet_manager.cc:1434] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.037s user 0.002s sys 0.005s
I20260926 19:58:48.053766 13012 tablet_bootstrap.cc:654] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:48.073364 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 9acbd4e5aa23433d860d399d78485d0d
I20260926 19:58:48.075099 12390 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:45734 (request call id 244) took 1066 ms. Trace:
I20260926 19:58:48.078830 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 9acbd4e5aa23433d860d399d78485d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:47 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:48.084065 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 9acbd4e5aa23433d860d399d78485d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:47 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:48.075457 12390 rpcz_store.cc:276] 0926 19:58:47.008735 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.104182 (+ 95447us) service_pool.cc:224] Handling call
0926 19:58:47.158393 (+ 54211us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8876 00007fb36a687b34 00007fb36a687ac2 00007fb36a6879f1 00007fb36a68796c 00007fb36a686c22 00007fb36892d09c 00007fb36892c4d6
0926 19:58:47.159752 (+ 1359us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:47.160230 (+ 478us) catalog_manager.cc:2854] Modifying in-memory table state
0926 19:58:47.160346 (+ 116us) catalog_manager.cc:2863] Locking tablets
0926 19:58:47.160959 (+ 613us) catalog_manager.cc:2877] Removing table and tablets from system table
0926 19:58:48.072896 (+911937us) catalog_manager.cc:2895] Removing table from by-name map
0926 19:58:48.073140 (+ 244us) catalog_manager.cc:2911] Committing in-memory state
0926 19:58:48.073261 (+ 121us) catalog_manager.cc:2915] Aborting table tasks
0926 19:58:48.075030 (+ 1769us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:47.906300 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:47.906722 (+ 422us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:47.906744 (+ 22us) write_op.cc:435] Acquired schema lock
0926 19:58:47.906755 (+ 11us) tablet.cc:662] Decoding operations
0926 19:58:47.907053 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:47.907106 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:47.907138 (+ 32us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:47.919593 (+ 12455us) tablet.cc:701] Row locks acquired
0926 19:58:47.919611 (+ 18us) write_op.cc:260] PREPARE: finished
0926 19:58:47.919723 (+ 112us) write_op.cc:270] Start()
0926 19:58:47.919805 (+ 82us) write_op.cc:276] Timestamp: P: 1790452727919671 usec, L: 0
0926 19:58:47.919819 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:47.920292 (+ 473us) log.cc:844] Serialized 774 byte log entry
0926 19:58:48.070598 (+150306us) op_driver.cc:464] REPLICATION: finished
0926 19:58:48.071215 (+ 617us) write_op.cc:301] APPLY: starting
0926 19:58:48.071282 (+ 67us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:48.071426 (+ 144us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:48.071549 (+ 123us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:48.071821 (+ 272us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:48.072023 (+ 202us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:58:48.072083 (+ 60us) write_op.cc:312] APPLY: finished
0926 19:58:48.072295 (+ 212us) log.cc:844] Serialized 29 byte log entry
0926 19:58:48.072447 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.072513 (+ 66us) write_op.cc:454] Released schema lock
0926 19:58:48.072541 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":122659584,"child_traces":[["op",{"apply.queue_time_us":442,"apply.run_cpu_time_us":1527,"apply.run_wall_time_us":1527,"num_ops":2,"prepare.queue_time_us":739913,"raft.queue_time_us":141766,"raft.run_cpu_time_us":924,"raft.run_wall_time_us":928,"replication_time_us":150661}]]}
I20260926 19:58:48.100638 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3ea5bfbeba4f471f9761aceba506dfa4" } new_table_name: "default.Table_24dedc75d9844c41976c575db604e1d7" modify_external_catalogs: true
I20260926 19:58:48.101464 13012 tablet_bootstrap.cc:492] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:48.101899 13012 ts_tablet_manager.cc:1403] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.067s user 0.004s sys 0.011s
I20260926 19:58:48.104066 13012 raft_consensus.cc:359] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.104655 13012 raft_consensus.cc:385] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:48.104895 13012 raft_consensus.cc:740] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:48.111326 12739 tablet_service.cc:1511] Processing CreateTablet for tablet f2663906e11549e3aa9ae78d88e645ea (DEFAULT_TABLE table=default.Table_447c2fd2552a48d0897da03fd0d1d963 [id=fcf0c9e6dcd647d09ad31bbc20abd068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:48.112339 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2663906e11549e3aa9ae78d88e645ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:48.113214 12898 tablet_service.cc:1511] Processing CreateTablet for tablet f2663906e11549e3aa9ae78d88e645ea (DEFAULT_TABLE table=default.Table_447c2fd2552a48d0897da03fd0d1d963 [id=fcf0c9e6dcd647d09ad31bbc20abd068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:48.114500 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2663906e11549e3aa9ae78d88e645ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:48.115566 13012 consensus_queue.cc:260] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:48.116258 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet d48ee26c0aaa4abc8e08a1a2dadb028d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d48ee26c0aaa4abc8e08a1a2dadb028d already in progress: creating tablet
W20260926 19:58:48.118598 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 8f59be98ceef4effa15baf487c2d1140 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8f59be98ceef4effa15baf487c2d1140 already in progress: creating tablet
I20260926 19:58:48.119656 13472 tablet_replica.cc:333] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:48.120554 13472 raft_consensus.cc:2243] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:48.121184 13472 raft_consensus.cc:2272] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:48.133057 12858 leader_election.cc:343] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 27ccbbe1a5784e97827a31e9febd8c20
I20260926 19:58:48.133473 12858 leader_election.cc:304] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:48.134428 13384 raft_consensus.cc:2749] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:48.152596 13466 raft_consensus.cc:493] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:48.135875 13401 raft_consensus.cc:697] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:48.178012 12519 leader_election.cc:304] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:48.178016 13401 consensus_queue.cc:237] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.136031 12594 raft_consensus.cc:3060] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.174690 12591 raft_consensus.cc:2468] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:48.216946 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 9acbd4e5aa23433d860d399d78485d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:47 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:48.218251 13477 raft_consensus.cc:2804] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:48.218183 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 36c14fcb0577402abe5fbb2645bd1856 (DEFAULT_TABLE table=default.Table_9ad397471def442f87f3594e163fe1fd [id=2b7641258c3f4856ad819887d6cf3bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:48.219492 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36c14fcb0577402abe5fbb2645bd1856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:48.222549 13477 raft_consensus.cc:493] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:48.222939 13477 raft_consensus.cc:3060] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.233912 13403 ts_tablet_manager.cc:1929] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:48.117409 13012 ts_tablet_manager.cc:1434] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.015s user 0.001s sys 0.003s
I20260926 19:58:48.280269 13012 tablet_bootstrap.cc:492] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:48.163914 12695 leader_election.cc:304] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
W20260926 19:58:48.210060 12696 consensus_peers.cc:597] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260926 19:58:48.281042 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 71132.8us
I20260926 19:58:48.288816 13012 tablet_bootstrap.cc:654] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:48.266482 13092 tablet_bootstrap.cc:492] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
W20260926 19:58:48.295522 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 131740us
I20260926 19:58:48.296223 12695 leader_election.cc:304] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:48.297153 13468 raft_consensus.cc:2804] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:48.277698 13384 raft_consensus.cc:493] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:48.297590 13468 raft_consensus.cc:697] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:48.297711 13409 raft_consensus.cc:2804] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:48.298072 13409 raft_consensus.cc:493] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:48.297777 13384 raft_consensus.cc:515] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.298373 13409 raft_consensus.cc:3060] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.298446 13468 consensus_queue.cc:237] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:48.299788 13384 leader_election.cc:290] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:48.305193 13409 raft_consensus.cc:515] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.306866 13472 ts_tablet_manager.cc:1916] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:48.309155 13092 tablet_bootstrap.cc:654] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:48.308889 13409 leader_election.cc:290] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:48.322446 12739 tablet_service.cc:1511] Processing CreateTablet for tablet e7a829c352f34e82b851563c93a24df8 (DEFAULT_TABLE table=default.Table_1e6fa952aacb4642abb1628901cdb74f [id=c9d1188f0bb544bcb00e047d51b92e60]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:48.322706 12382 consensus_peers.cc:597] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 -> Peer db2490adb6e84090b780f52470e580ab (127.11.233.253:37913): Couldn't send request to peer db2490adb6e84090b780f52470e580ab. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260926 19:58:48.332422 13403 log.cc:1199] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/3215bbd5be56438491a73925ac9ccbe6
I20260926 19:58:48.332693 13012 tablet_bootstrap.cc:492] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:48.333173 13012 ts_tablet_manager.cc:1403] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.053s user 0.006s sys 0.005s
I20260926 19:58:48.336161 13012 raft_consensus.cc:359] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.337061 13012 raft_consensus.cc:385] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:48.338280 13403 ts_tablet_manager.cc:1950] T 3215bbd5be56438491a73925ac9ccbe6 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:48.341882 13384 raft_consensus.cc:493] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:48.342428 13384 raft_consensus.cc:515] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.345831 13384 leader_election.cc:290] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:48.355669 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:48.356472 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_b68b74a1774041749a10f4006f15ece1 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 1 got 0
I20260926 19:58:48.337400 13012 raft_consensus.cc:740] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:48.357290 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 64149f9e4d384ddbb922291ac7de35b7 (table default.Table_c489e926efca4dafb56b6642f8404c2b [id=00773dee014a47fca49e7dfd9ed0d584]). Expected version 2 got 0
I20260926 19:58:48.359671 12388 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_c489e926efca4dafb56b6642f8404c2b [id=00773dee014a47fca49e7dfd9ed0d584] alter complete (version 2)
I20260926 19:58:48.360687 12392 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:45734 (request call id 245) took 1255 ms. Trace:
I20260926 19:58:48.360225 13012 consensus_queue.cc:260] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:48.361860 12388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 51) took 1778 ms (client timeout 1000 ms). Trace:
W20260926 19:58:48.362195 12388 rpcz_store.cc:269] 0926 19:58:46.583288 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.589594 (+ 6306us) service_pool.cc:224] Handling call
0926 19:58:48.361798 (+1772204us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:47.770135 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:47.770483 (+ 348us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:47.770505 (+ 22us) write_op.cc:435] Acquired schema lock
0926 19:58:47.770517 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:47.770743 (+ 226us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:47.770777 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:47.770799 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:47.770931 (+ 132us) tablet.cc:701] Row locks acquired
0926 19:58:47.770944 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:47.771066 (+ 122us) write_op.cc:270] Start()
0926 19:58:47.771145 (+ 79us) write_op.cc:276] Timestamp: P: 1790452727771020 usec, L: 0
0926 19:58:47.771160 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:47.861289 (+ 90129us) log.cc:844] Serialized 356 byte log entry
0926 19:58:47.885815 (+ 24526us) op_driver.cc:464] REPLICATION: finished
0926 19:58:47.886678 (+ 863us) write_op.cc:301] APPLY: starting
0926 19:58:47.886737 (+ 59us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:47.886820 (+ 83us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:47.886925 (+ 105us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:47.887103 (+ 178us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:47.887241 (+ 138us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:47.887281 (+ 40us) write_op.cc:312] APPLY: finished
0926 19:58:47.887457 (+ 176us) log.cc:844] Serialized 23 byte log entry
0926 19:58:47.904225 (+ 16768us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:47.904290 (+ 65us) write_op.cc:454] Released schema lock
0926 19:58:47.904329 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:47.906863 (+ 2534us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:47.906888 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":653,"apply.run_cpu_time_us":1344,"apply.run_wall_time_us":20206,"num_ops":1,"prepare.queue_time_us":654745,"prepare.run_cpu_time_us":1865,"prepare.run_wall_time_us":136016,"raft.queue_time_us":1278,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":257,"replication_time_us":114552,"spinlock_wait_cycles":2304}]]}
I20260926 19:58:48.361013 12392 rpcz_store.cc:276] 0926 19:58:47.105762 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.397869 (+292107us) service_pool.cc:224] Handling call
0926 19:58:47.416062 (+ 18193us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8be6 00007fb36a68c634 00007fb36a68c5c2 00007fb36a68c4f1 00007fb36a68c46c 00007fb36a68b722 00007fb36892d09c 00007fb36892c4d6
0926 19:58:47.417067 (+ 1005us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:47.418667 (+ 1600us) catalog_manager.cc:3764] Apply alter schema
0926 19:58:47.419400 (+ 733us) catalog_manager.cc:3796] Acquired catalog manager lock
0926 19:58:47.419627 (+ 227us) catalog_manager.cc:3954] Updating metadata on disk
0926 19:58:48.245603 (+825976us) catalog_manager.cc:3985] Committing alterations to in-memory state
0926 19:58:48.360625 (+115022us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:48.097901 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:48.098405 (+ 504us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:48.098436 (+ 31us) write_op.cc:435] Acquired schema lock
0926 19:58:48.098455 (+ 19us) tablet.cc:662] Decoding operations
0926 19:58:48.098748 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:48.098801 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:48.098833 (+ 32us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:48.099017 (+ 184us) tablet.cc:701] Row locks acquired
0926 19:58:48.099036 (+ 19us) write_op.cc:260] PREPARE: finished
0926 19:58:48.099160 (+ 124us) write_op.cc:270] Start()
0926 19:58:48.099260 (+ 100us) write_op.cc:276] Timestamp: P: 1790452728099110 usec, L: 0
0926 19:58:48.099282 (+ 22us) op_driver.cc:348] REPLICATION: starting
0926 19:58:48.099843 (+ 561us) log.cc:844] Serialized 405 byte log entry
0926 19:58:48.243581 (+143738us) op_driver.cc:464] REPLICATION: finished
0926 19:58:48.244023 (+ 442us) write_op.cc:301] APPLY: starting
0926 19:58:48.244098 (+ 75us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:48.244213 (+ 115us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:48.244337 (+ 124us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:48.244555 (+ 218us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:48.244724 (+ 169us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:48.244784 (+ 60us) write_op.cc:312] APPLY: finished
0926 19:58:48.245005 (+ 221us) log.cc:844] Serialized 23 byte log entry
0926 19:58:48.245154 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.245213 (+ 59us) write_op.cc:454] Released schema lock
0926 19:58:48.245241 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:48.254184 (+ 8943us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.254219 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":38386560,"child_traces":[["op",{"apply.queue_time_us":263,"apply.run_cpu_time_us":1411,"apply.run_wall_time_us":1410,"num_ops":1,"prepare.queue_time_us":676653,"prepare.run_cpu_time_us":2295,"prepare.run_wall_time_us":156256,"raft.queue_time_us":135405,"raft.run_cpu_time_us":941,"raft.run_wall_time_us":940,"replication_time_us":144178}]]}
I20260926 19:58:48.333663 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:48.372714 12696 leader_election.cc:343] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:48.374518 13012 ts_tablet_manager.cc:1434] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.041s user 0.004s sys 0.004s
I20260926 19:58:48.378317 12567 tablet_service.cc:1511] Processing CreateTablet for tablet f2663906e11549e3aa9ae78d88e645ea (DEFAULT_TABLE table=default.Table_447c2fd2552a48d0897da03fd0d1d963 [id=fcf0c9e6dcd647d09ad31bbc20abd068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:48.379601 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2663906e11549e3aa9ae78d88e645ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:48.381683 12519 consensus_peers.cc:597] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260926 19:58:48.382074 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 154254us
I20260926 19:58:48.223796 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 165) took 2124 ms. Trace:
I20260926 19:58:48.385838 13203 rpcz_store.cc:276] 0926 19:58:46.099819 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.103041 (+ 3222us) service_pool.cc:224] Handling call
0926 19:58:47.519281 (+1416240us) ts_tablet_manager.cc:1151] Deleting tablet ba3f3931a71048fd874c1131d44f7b00
0926 19:58:47.519321 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:47.534888 (+ 15567us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:47.852442 (+317554us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:48.223728 (+371286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7807,"tablet-delete.queue_time_us":1415696}
I20260926 19:58:48.387274 13203 tablet_replica.cc:333] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
W20260926 19:58:48.399910 12858 leader_election.cc:343] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 1455b99a65ff44ff933825196708f485
W20260926 19:58:48.400512 12858 leader_election.cc:343] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: cd92a1a62ab24647bef7c9eefdf9c9d8
I20260926 19:58:48.409830 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet ba3f3931a71048fd874c1131d44f7b00 (table default.Table_a34ea809f74841d18b133415daf8192a [id=3dd781a8406045b6b90c7008824c7a06]) successfully deleted
I20260926 19:58:48.422339 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:48.458348 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:48.459353 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_f136506f88074adcb77c8ab2bc88ce51 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 2 got 0
I20260926 19:58:48.460770 12392 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:48.469276 12900 tablet_service.cc:1511] Processing CreateTablet for tablet e7a829c352f34e82b851563c93a24df8 (DEFAULT_TABLE table=default.Table_1e6fa952aacb4642abb1628901cdb74f [id=c9d1188f0bb544bcb00e047d51b92e60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:48.470652 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a829c352f34e82b851563c93a24df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:48.496457 13472 ts_tablet_manager.cc:1929] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:48.497026 13472 log.cc:1199] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/9acbd4e5aa23433d860d399d78485d0d
I20260926 19:58:48.498119 13472 ts_tablet_manager.cc:1950] T 9acbd4e5aa23433d860d399d78485d0d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:48.176862 13466 raft_consensus.cc:515] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.505993 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd92a1a62ab24647bef7c9eefdf9c9d8" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:48.506636 12757 raft_consensus.cc:2468] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:48.514536 12857 leader_election.cc:304] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:48.516338 13384 raft_consensus.cc:2804] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:48.516824 13384 raft_consensus.cc:493] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:48.517251 13384 raft_consensus.cc:3060] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.535434 13384 raft_consensus.cc:515] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.542392 12561 tablet_service.cc:1511] Processing CreateTablet for tablet e7a829c352f34e82b851563c93a24df8 (DEFAULT_TABLE table=default.Table_1e6fa952aacb4642abb1628901cdb74f [id=c9d1188f0bb544bcb00e047d51b92e60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:48.543429 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a829c352f34e82b851563c93a24df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:48.551249 13501 tablet.cc:1751] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:48.552457 13012 tablet_bootstrap.cc:492] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:48.564054 13479 log.cc:927] Time spent T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Append to log took a long time: real 0.064s user 0.001s sys 0.002s
I20260926 19:58:48.565268 13409 consensus_queue.cc:1048] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:48.566692 13012 tablet_bootstrap.cc:654] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:48.572846 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 9acbd4e5aa23433d860d399d78485d0d (table default.Table_e0ad88ad494c42759968cb1dc8501282 [id=6890560cf0104034a9eb9431a38c99ec]) successfully deleted
I20260926 19:58:48.581954 13013 tablet_bootstrap.cc:492] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:48.584398 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e337767be84507acdecfb189705bdd" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:48.584546 13466 leader_election.cc:290] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:48.585884 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e337767be84507acdecfb189705bdd" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:48.586485 12757 raft_consensus.cc:2468] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:48.586886 12594 raft_consensus.cc:2468] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:48.587802 13013 ts_tablet_manager.cc:1403] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 1.061s user 0.011s sys 0.010s
I20260926 19:58:48.589327 12519 leader_election.cc:304] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:48.594516 12858 leader_election.cc:304] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:48.595767 13426 raft_consensus.cc:2804] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
W20260926 19:58:48.606621 12521 leader_election.cc:343] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:48.608253 13448 raft_consensus.cc:2804] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:48.608671 13448 raft_consensus.cc:493] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:48.608986 13448 raft_consensus.cc:3060] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.617949 12388 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 732a34f4894940b3889bab5f0c0891e0 (table default.Table_f136506f88074adcb77c8ab2bc88ce51 [id=80c38f5ed2d841bc82af02a3dc13ba60]). Expected version 2 got 0
I20260926 19:58:48.606626 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1455b99a65ff44ff933825196708f485" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:48.507408 13092 tablet_bootstrap.cc:492] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:48.627652 13013 raft_consensus.cc:359] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.628306 13013 raft_consensus.cc:385] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:48.631368 13013 raft_consensus.cc:740] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:48.632278 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7a829c352f34e82b851563c93a24df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:48.631978 13013 consensus_queue.cc:260] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.634837 13013 ts_tablet_manager.cc:1434] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.047s user 0.004s sys 0.002s
I20260926 19:58:48.635749 13013 tablet_bootstrap.cc:492] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
W20260926 19:58:48.639263 12380 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet fcad0bbb2bb44338bd56f2547e70ab9c consensus state has no leader
I20260926 19:58:48.640983 13092 ts_tablet_manager.cc:1403] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.375s user 0.013s sys 0.000s
I20260926 19:58:48.642135 13203 raft_consensus.cc:2243] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:48.650563 13477 raft_consensus.cc:515] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.651643 12595 raft_consensus.cc:2393] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:48.653569 13477 leader_election.cc:290] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:48.655201 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:48.655951 12757 raft_consensus.cc:2393] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f6e9d6439fd54b28925a68a6145d3dee in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:48.666320 13448 raft_consensus.cc:515] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.619357 12388 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:48.684574 12695 leader_election.cc:304] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:48.686278 13409 raft_consensus.cc:2749] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:48.690361 13448 leader_election.cc:290] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:48.699290 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e337767be84507acdecfb189705bdd" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:48.699862 12757 raft_consensus.cc:3060] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.701627 13501 tablet.cc:1751] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:48.707932 13511 tablet.cc:1751] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:48.677642 13203 raft_consensus.cc:2272] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:48.714123 13501 tablet.cc:1751] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:48.723263 13448 raft_consensus.cc:493] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:48.723712 13448 raft_consensus.cc:515] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.728335 13448 leader_election.cc:290] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:48.731236 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dabe03472b640eca7d88e4c54cc2782" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:48.731817 12760 raft_consensus.cc:2468] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:48.732807 12519 leader_election.cc:304] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:48.732901 13426 raft_consensus.cc:697] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:48.733557 13448 raft_consensus.cc:2804] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:48.733883 13448 raft_consensus.cc:493] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:48.734177 13448 raft_consensus.cc:3060] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.733803 13426 consensus_queue.cc:237] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.747350 12757 raft_consensus.cc:2468] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:48.748594 12519 leader_election.cc:304] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:48.749589 13466 raft_consensus.cc:2804] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:48.750496 13466 raft_consensus.cc:697] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:48.751240 13466 consensus_queue.cc:237] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.772236 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e337767be84507acdecfb189705bdd" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:48.773780 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dabe03472b640eca7d88e4c54cc2782" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:48.775218 12395 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45734 (request call id 246) took 1342 ms. Trace:
W20260926 19:58:48.775358 12521 leader_election.cc:343] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:48.776152 12521 leader_election.cc:343] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:48.729122 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 157) took 2415 ms. Trace:
I20260926 19:58:48.780138 12388 catalog_manager.cc:5838] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:48.781759 12388 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]). Expected version 2 got 0
I20260926 19:58:48.782011 13403 rpcz_store.cc:276] 0926 19:58:46.260031 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.260158 (+ 127us) service_pool.cc:224] Handling call
0926 19:58:47.494592 (+1234434us) ts_tablet_manager.cc:1151] Deleting tablet 3215bbd5be56438491a73925ac9ccbe6
0926 19:58:47.494634 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:47.546898 (+ 52264us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b54000c3854. stack: 00007ffa5c912323 00007ffa5c912220 00007ffa5c4b6574 0000000000516964 00000000005168f9 00007ffa63a0faa1 00007ffa63aaeb22 00007ffa63eef55c 00007ffa64b58c3a 00007ffa64b89960 00007ffa64b6fe6d 00007ffa64b6fe19 00007ffa64b6fda9 00007ffa64b6fd71 00007ffa64b6ef9d 00000000005bd4e4
0926 19:58:47.761953 (+215055us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:48.233756 (+471803us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:48.675494 (+441738us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113031936,"tablet-delete.queue_time_us":1233906}
I20260926 19:58:48.686926 13509 tablet.cc:1751] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:48.622437 12756 raft_consensus.cc:2468] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:48.783488 13403 tablet_replica.cc:333] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:48.784292 12857 leader_election.cc:304] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:48.780833 13499 tablet.cc:1751] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:48.781577 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:48.788681 13432 raft_consensus.cc:493] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:48.789143 13432 raft_consensus.cc:515] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.789537 13498 raft_consensus.cc:2804] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:48.789846 13498 raft_consensus.cc:493] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:48.790161 13498 raft_consensus.cc:3060] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.791494 13432 leader_election.cc:290] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:48.800779 13403 raft_consensus.cc:2243] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:48.810072 13403 raft_consensus.cc:2272] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:48.795275 13092 raft_consensus.cc:359] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.811012 13092 raft_consensus.cc:385] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:48.811324 13092 raft_consensus.cc:740] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:48.677759 13013 tablet_bootstrap.cc:654] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:48.789036 12521 leader_election.cc:343] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:48.816622 12521 leader_election.cc:304] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:48.812124 13092 consensus_queue.cc:260] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.801222 13448 raft_consensus.cc:515] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.822027 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dabe03472b640eca7d88e4c54cc2782" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:48.822575 12757 raft_consensus.cc:3060] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.825129 13092 ts_tablet_manager.cc:1434] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.184s user 0.005s sys 0.000s
I20260926 19:58:48.825708 13403 ts_tablet_manager.cc:1916] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:48.827518 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:48.828579 13092 tablet_bootstrap.cc:492] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:48.829171 12757 raft_consensus.cc:2468] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:48.809092 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:48.825604 13448 leader_election.cc:290] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:48.829988 12595 raft_consensus.cc:2393] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in this term.
W20260926 19:58:48.830380 13494 log.cc:927] Time spent T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: Append to log took a long time: real 0.323s user 0.002s sys 0.000s
I20260926 19:58:48.829931 13448 raft_consensus.cc:2749] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:48.831444 12695 leader_election.cc:304] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:48.834052 13432 raft_consensus.cc:3060] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.833974 12519 leader_election.cc:304] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:48.834745 13448 raft_consensus.cc:2804] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:48.835186 13448 raft_consensus.cc:697] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:48.835968 13448 consensus_queue.cc:237] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.832713 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dabe03472b640eca7d88e4c54cc2782" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:48.838615 12521 leader_election.cc:343] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:48.844772 13504 tablet.cc:1751] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:48.846835 13498 raft_consensus.cc:515] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:48.851939 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 220780us
I20260926 19:58:48.852870 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1455b99a65ff44ff933825196708f485" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:48.853430 12757 raft_consensus.cc:3060] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.862493 13012 tablet_bootstrap.cc:492] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:48.863119 13012 ts_tablet_manager.cc:1403] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.311s user 0.012s sys 0.001s
I20260926 19:58:48.870018 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 3215bbd5be56438491a73925ac9ccbe6 (table default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]) successfully deleted
I20260926 19:58:48.888652 12757 raft_consensus.cc:2468] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:48.910171 12857 leader_election.cc:304] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:48.911653 13506 raft_consensus.cc:2804] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:48.912268 13506 raft_consensus.cc:697] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:48.913414 13506 consensus_queue.cc:237] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.782698 12388 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (table default.Table_653e7d4c6df34da98c75d496cee46f27 [id=c2c079dc87ec480a82aa5b57b110da17]). Expected version 2 got 0
I20260926 19:58:48.918505 12388 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:48.919660 13432 raft_consensus.cc:2749] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:48.923216 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 68245.8us
W20260926 19:58:48.923902 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 61354.1us
I20260926 19:58:48.916481 13384 leader_election.cc:290] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:48.920815 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd92a1a62ab24647bef7c9eefdf9c9d8" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:48.958185 12756 raft_consensus.cc:3060] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:48.966850 12756 raft_consensus.cc:2468] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:48.971369 13013 tablet_bootstrap.cc:492] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:48.971822 13013 ts_tablet_manager.cc:1403] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.336s user 0.014s sys 0.021s
W20260926 19:58:48.974216 12858 leader_election.cc:343] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 1455b99a65ff44ff933825196708f485
I20260926 19:58:48.974498 13511 tablet.cc:1751] T 732a34f4894940b3889bab5f0c0891e0 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260926 19:58:48.974900 12858 leader_election.cc:343] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: cd92a1a62ab24647bef7c9eefdf9c9d8
I20260926 19:58:48.775597 12395 rpcz_store.cc:276] 0926 19:58:47.432764 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.440339 (+ 7575us) service_pool.cc:224] Handling call
0926 19:58:47.474618 (+ 34279us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8566 00007fb36a683034 00007fb36a682fc2 00007fb36a682ef1 00007fb36a682e6c 00007fb36a682122 00007fb36892d09c 00007fb36892c4d6
0926 19:58:47.477326 (+ 2708us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:58:47.479962 (+ 2636us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:58:48.661727 (+1181765us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:58:48.774860 (+113133us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:58:48.775154 (+ 294us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:48.462736 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:48.463202 (+ 466us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:48.463232 (+ 30us) write_op.cc:435] Acquired schema lock
0926 19:58:48.463365 (+ 133us) tablet.cc:662] Decoding operations
0926 19:58:48.463677 (+ 312us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:48.463727 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:48.463758 (+ 31us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:48.463987 (+ 229us) tablet.cc:701] Row locks acquired
0926 19:58:48.464005 (+ 18us) write_op.cc:260] PREPARE: finished
0926 19:58:48.464160 (+ 155us) write_op.cc:270] Start()
0926 19:58:48.464254 (+ 94us) write_op.cc:276] Timestamp: P: 1790452728464096 usec, L: 0
0926 19:58:48.464273 (+ 19us) op_driver.cc:348] REPLICATION: starting
0926 19:58:48.464844 (+ 571us) log.cc:844] Serialized 433 byte log entry
0926 19:58:48.658949 (+194105us) op_driver.cc:464] REPLICATION: finished
0926 19:58:48.659350 (+ 401us) write_op.cc:301] APPLY: starting
0926 19:58:48.659409 (+ 59us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:48.659524 (+ 115us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:48.659620 (+ 96us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:48.660057 (+ 437us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:48.660234 (+ 177us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:58:48.660309 (+ 75us) write_op.cc:312] APPLY: finished
0926 19:58:48.660527 (+ 218us) log.cc:844] Serialized 29 byte log entry
0926 19:58:48.660684 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.660765 (+ 81us) write_op.cc:454] Released schema lock
0926 19:58:48.660805 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:48.696223 (+ 35418us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.696284 (+ 61us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":76639488,"child_traces":[["op",{"apply.queue_time_us":246,"apply.run_cpu_time_us":1671,"apply.run_wall_time_us":1670,"num_ops":2,"prepare.queue_time_us":981143,"prepare.run_cpu_time_us":2471,"prepare.run_wall_time_us":233409,"raft.queue_time_us":177444,"raft.run_cpu_time_us":858,"raft.run_wall_time_us":858,"replication_time_us":194523}]]}
I20260926 19:58:48.930460 12394 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45734 (request call id 247) took 1328 ms. Trace:
W20260926 19:58:48.977674 13494 consensus_queue.cc:1175] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f6e9d6439fd54b28925a68a6145d3dee" status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260926 19:58:48.979075 13501 tablet.cc:1751] T 732a34f4894940b3889bab5f0c0891e0 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:48.977864 12394 rpcz_store.cc:276] 0926 19:58:47.602016 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.602182 (+ 166us) service_pool.cc:224] Handling call
0926 19:58:47.610174 (+ 7992us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8566 00007fb36a683034 00007fb36a682fc2 00007fb36a682ef1 00007fb36a682e6c 00007fb36a682122 00007fb36892d09c 00007fb36892c4d6
0926 19:58:47.612258 (+ 2084us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:58:47.614486 (+ 2228us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:58:48.929647 (+1315161us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:58:48.930031 (+ 384us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:58:48.930382 (+ 351us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:48.696766 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:48.785852 (+ 89086us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:48.785922 (+ 70us) write_op.cc:435] Acquired schema lock
0926 19:58:48.785935 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:48.786198 (+ 263us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:48.786237 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:48.786259 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:48.786441 (+ 182us) tablet.cc:701] Row locks acquired
0926 19:58:48.786454 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:48.786577 (+ 123us) write_op.cc:270] Start()
0926 19:58:48.786668 (+ 91us) write_op.cc:276] Timestamp: P: 1790452728786532 usec, L: 0
0926 19:58:48.786690 (+ 22us) op_driver.cc:348] REPLICATION: starting
0926 19:58:48.787171 (+ 481us) log.cc:844] Serialized 433 byte log entry
0926 19:58:48.925567 (+138396us) op_driver.cc:464] REPLICATION: finished
0926 19:58:48.927306 (+ 1739us) write_op.cc:301] APPLY: starting
0926 19:58:48.927393 (+ 87us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:48.927543 (+ 150us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:48.927682 (+ 139us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:48.928254 (+ 572us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:48.928462 (+ 208us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:58:48.928530 (+ 68us) write_op.cc:312] APPLY: finished
0926 19:58:48.928784 (+ 254us) log.cc:844] Serialized 29 byte log entry
0926 19:58:48.928976 (+ 192us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.929064 (+ 88us) write_op.cc:454] Released schema lock
0926 19:58:48.929106 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:48.929403 (+ 297us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.929438 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":16485248,"child_traces":[["op",{"apply.queue_time_us":1567,"apply.run_cpu_time_us":2039,"apply.run_wall_time_us":2039,"num_ops":2,"prepare.queue_time_us":1080749,"prepare.run_cpu_time_us":2263,"prepare.run_wall_time_us":180926,"raft.queue_time_us":138,"raft.run_cpu_time_us":990,"raft.run_wall_time_us":2126,"replication_time_us":138749}]]}
I20260926 19:58:48.934702 12396 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45734 (request call id 248) took 1288 ms. Trace:
I20260926 19:58:48.981282 12396 rpcz_store.cc:276] 0926 19:58:47.646394 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.647621 (+ 1227us) service_pool.cc:224] Handling call
0926 19:58:47.656248 (+ 8627us) spinlock_profiling.cc:243] Waited 7.61 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8566 00007fb36a683034 00007fb36a682fc2 00007fb36a682ef1 00007fb36a682e6c 00007fb36a682122 00007fb36892d09c 00007fb36892c4d6
0926 19:58:47.663890 (+ 7642us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:58:47.675077 (+ 11187us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:58:48.933877 (+1258800us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:58:48.934276 (+ 399us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:58:48.934626 (+ 350us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:48.877740 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:48.878085 (+ 345us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:48.878108 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:58:48.878121 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:48.878418 (+ 297us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:48.878484 (+ 66us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:48.878519 (+ 35us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:48.878763 (+ 244us) tablet.cc:701] Row locks acquired
0926 19:58:48.878789 (+ 26us) write_op.cc:260] PREPARE: finished
0926 19:58:48.878933 (+ 144us) write_op.cc:270] Start()
0926 19:58:48.879037 (+ 104us) write_op.cc:276] Timestamp: P: 1790452728878874 usec, L: 0
0926 19:58:48.879059 (+ 22us) op_driver.cc:348] REPLICATION: starting
0926 19:58:48.879697 (+ 638us) log.cc:844] Serialized 433 byte log entry
0926 19:58:48.926743 (+ 47046us) op_driver.cc:464] REPLICATION: finished
0926 19:58:48.931910 (+ 5167us) write_op.cc:301] APPLY: starting
0926 19:58:48.931962 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:48.932089 (+ 127us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:48.932212 (+ 123us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:48.932785 (+ 573us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:48.932983 (+ 198us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:58:48.933045 (+ 62us) write_op.cc:312] APPLY: finished
0926 19:58:48.933290 (+ 245us) log.cc:844] Serialized 29 byte log entry
0926 19:58:48.933482 (+ 192us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.933595 (+ 113us) write_op.cc:454] Released schema lock
0926 19:58:48.933642 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:48.970289 (+ 36647us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.970317 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":17512704,"child_traces":[["op",{"apply.queue_time_us":5019,"apply.run_cpu_time_us":1985,"apply.run_wall_time_us":3363,"num_ops":2,"prepare.queue_time_us":1201192,"prepare.run_cpu_time_us":2336,"prepare.run_wall_time_us":92522,"raft.queue_time_us":99,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":259,"replication_time_us":47562,"thread_start_us":29384,"threads_started":1}]]}
I20260926 19:58:48.990018 12593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.11.233.194:46173 (request call id 99) took 1051 ms. Trace:
I20260926 19:58:48.988356 12857 leader_election.cc:304] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:48.990355 12593 rpcz_store.cc:276] 0926 19:58:47.938197 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.938852 (+ 655us) service_pool.cc:224] Handling call
0926 19:58:47.961115 (+ 22263us) raft_consensus.cc:1471] Updating replica for 1 ops
0926 19:58:48.412905 (+451790us) raft_consensus.cc:1534] Early marking committed up to index 0
0926 19:58:48.423352 (+ 10447us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0926 19:58:48.423867 (+ 515us) log.cc:844] Serialized 26 byte log entry
0926 19:58:48.503592 (+ 79725us) raft_consensus.cc:1645] Marking committed up to 1
0926 19:58:48.621839 (+118247us) log.cc:844] Serialized 14 byte log entry
0926 19:58:48.641987 (+ 20148us) raft_consensus.cc:1697] Filling consensus response to leader.
0926 19:58:48.642745 (+ 758us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0926 19:58:48.700645 (+ 57900us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b440003484c. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a1e5688a1 00007f9a1e606775 00007f9a1e607192 00007f9a256283bb 00007f9a25623c9f 00007f9a25617a5e 00007f9a25616da6 00007f9a256169c8 00007f9a25620411 00007f9a256203c9 00007f9a25620359
0926 19:58:48.989692 (+289047us) raft_consensus.cc:1685] finished
0926 19:58:48.989814 (+ 122us) raft_consensus.cc:1691] UpdateReplicas() finished
0926 19:58:48.989952 (+ 138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38002,"prepare.queue_time_us":119411,"prepare.run_cpu_time_us":11,"prepare.run_wall_time_us":12,"raft.queue_time_us":177926,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":268,"spinlock_wait_cycles":300574592,"thread_start_us":492650,"threads_started":2,"wal-append.queue_time_us":78729}
I20260926 19:58:48.992362 13514 raft_consensus.cc:2804] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
W20260926 19:58:48.991878 12519 consensus_peers.cc:597] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3215bbd5be56438491a73925ac9ccbe6. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:48.992830 13514 raft_consensus.cc:697] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:48.993662 13514 consensus_queue.cc:237] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:48.995249 13203 ts_tablet_manager.cc:1916] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:48.996477 13013 raft_consensus.cc:359] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.000147 13464 raft_consensus.cc:493] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:49.000563 13464 raft_consensus.cc:515] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.002035 13499 tablet.cc:1751] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:49.003227 13464 leader_election.cc:290] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:49.005565 13384 raft_consensus.cc:493] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:49.005980 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:49.006039 13384 raft_consensus.cc:515] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.006642 12756 raft_consensus.cc:2468] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:49.007622 12519 leader_election.cc:304] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:49.008322 13464 raft_consensus.cc:2804] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 19:58:49.008615 13464 raft_consensus.cc:493] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:49.008924 13464 raft_consensus.cc:3060] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:49.014856 13013 raft_consensus.cc:385] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.015208 13013 raft_consensus.cc:740] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.015977 13013 consensus_queue.cc:260] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.019049 13013 ts_tablet_manager.cc:1434] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.047s user 0.005s sys 0.003s
I20260926 19:58:49.020637 13013 tablet_bootstrap.cc:492] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:49.024710 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189eaca5fb2b4b0d83a2873e0495e5ff" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:49.025972 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "189eaca5fb2b4b0d83a2873e0495e5ff" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:49.027683 12857 leader_election.cc:304] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:49.028502 13506 raft_consensus.cc:2749] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:49.032730 13464 raft_consensus.cc:515] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.033330 13384 leader_election.cc:290] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:49.035022 13499 tablet.cc:1751] T 732a34f4894940b3889bab5f0c0891e0 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:49.042227 13013 tablet_bootstrap.cc:654] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:49.060536 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:49.061053 12924 raft_consensus.cc:3060] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:49.062602 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:49.069801 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53668c46bd3441ad936bf990b3192420" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:49.070348 12756 raft_consensus.cc:3060] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:49.056572 13509 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37792 (request call id 178) took 1406 ms. Trace:
I20260926 19:58:49.079572 13509 rpcz_store.cc:276] 0926 19:58:47.649888 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.677729 (+ 27841us) service_pool.cc:224] Handling call
0926 19:58:49.056503 (+1378774us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:47.958324 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0926 19:58:47.959141 (+ 817us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0926 19:58:47.959196 (+ 55us) alter_schema_op.cc:66] Acquired schema lock
0926 19:58:47.959278 (+ 82us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0926 19:58:47.959424 (+ 146us) alter_schema_op.cc:128] START. Timestamp: 1790452727959313
0926 19:58:47.959466 (+ 42us) op_driver.cc:348] REPLICATION: starting
0926 19:58:47.960270 (+ 804us) log.cc:844] Serialized 173 byte log entry
0926 19:58:48.563398 (+603128us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b440003484c. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a1e5688a1 00007f9a1e606775 00007f9a1e607192 00007f9a256283bb 00007f9a25623c9f 00007f9a25617a5e 00007f9a25616da6 00007f9a256169c8 00007f9a25620411 00007f9a256203c9 00007f9a25620359
0926 19:58:48.614666 (+ 51268us) op_driver.cc:464] REPLICATION: finished
0926 19:58:48.686701 (+ 72035us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0926 19:58:48.813126 (+126425us) log.cc:844] Serialized 5 byte log entry
0926 19:58:49.044079 (+230953us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.044855 (+ 776us) log.cc:844] Serialized 14 byte log entry
0926 19:58:49.056355 (+ 11500us) alter_schema_op.cc:73] Released schema lock
0926 19:58:49.056386 (+ 31us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":71784,"mutex_wait_us":15689,"prepare.queue_time_us":279320,"prepare.run_cpu_time_us":15301,"prepare.run_wall_time_us":543887,"raft.queue_time_us":35303,"raft.run_cpu_time_us":15489,"raft.run_wall_time_us":93205,"replication_time_us":648324,"spinlock_wait_cycles":8998784,"thread_start_us":745241,"threads_started":3,"wal-append.queue_time_us":539511}]]}
I20260926 19:58:49.093637 12756 raft_consensus.cc:2468] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 2.
I20260926 19:58:49.094831 12519 leader_election.cc:304] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:49.096395 13477 raft_consensus.cc:2804] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:58:49.096856 13477 raft_consensus.cc:697] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [term 2 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:49.097631 13477 consensus_queue.cc:237] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.104276 13012 raft_consensus.cc:359] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.138068 13012 raft_consensus.cc:385] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.138571 13012 raft_consensus.cc:740] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.109764 13409 consensus_queue.cc:1048] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:49.139889 13012 consensus_queue.cc:260] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.111084 13498 leader_election.cc:290] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:49.114460 13464 leader_election.cc:290] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:49.128126 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c489e926efca4dafb56b6642f8404c2b" } modify_external_catalogs: true
I20260926 19:58:49.141292 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c489e926efca4dafb56b6642f8404c2b" } modify_external_catalogs: true
I20260926 19:58:49.129733 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a9b6637851b849c38be655d4ad6e9a5c" } modify_external_catalogs: true
I20260926 19:58:49.144021 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a9b6637851b849c38be655d4ad6e9a5c" } modify_external_catalogs: true
I20260926 19:58:49.148080 13092 tablet_bootstrap.cc:654] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:49.151881 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 59f7dc92330d400f93f427f3f79ad826
I20260926 19:58:49.118726 12924 raft_consensus.cc:2468] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 2.
I20260926 19:58:49.153283 12387 catalog_manager.cc:6934] ReplaceTablet: tablet 59f7dc92330d400f93f427f3f79ad826 deleted and replaced by tablet 4d153e60ae5d486cb91c2cc623b99985
I20260926 19:58:49.153600 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 59f7dc92330d400f93f427f3f79ad826 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d153e60ae5d486cb91c2cc623b99985) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:49.153647 12387 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:45734 (request call id 249) took 1491 ms. Trace:
I20260926 19:58:49.130606 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_653e7d4c6df34da98c75d496cee46f27" } new_table_name: "default.Table_d6b47ef12b28492bb711ca846f80f6cb" modify_external_catalogs: true
I20260926 19:58:49.153937 12387 rpcz_store.cc:276] 0926 19:58:47.662663 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.886148 (+223485us) service_pool.cc:224] Handling call
0926 19:58:49.153592 (+1267444us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:48.970581 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:48.970965 (+ 384us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:48.970989 (+ 24us) write_op.cc:435] Acquired schema lock
0926 19:58:48.971001 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:48.971435 (+ 434us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:48.971479 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:48.971505 (+ 26us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:48.971810 (+ 305us) tablet.cc:701] Row locks acquired
0926 19:58:48.971824 (+ 14us) write_op.cc:260] PREPARE: finished
0926 19:58:48.972142 (+ 318us) write_op.cc:270] Start()
0926 19:58:48.972224 (+ 82us) write_op.cc:276] Timestamp: P: 1790452728972088 usec, L: 0
0926 19:58:48.972239 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:48.972726 (+ 487us) log.cc:844] Serialized 631 byte log entry
0926 19:58:49.146661 (+173935us) op_driver.cc:464] REPLICATION: finished
0926 19:58:49.148126 (+ 1465us) write_op.cc:301] APPLY: starting
0926 19:58:49.148182 (+ 56us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:49.148299 (+ 117us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:49.148429 (+ 130us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:49.148774 (+ 345us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:49.149010 (+ 236us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:49.149058 (+ 48us) write_op.cc:312] APPLY: finished
0926 19:58:49.149250 (+ 192us) log.cc:844] Serialized 29 byte log entry
0926 19:58:49.149424 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.149490 (+ 66us) write_op.cc:454] Released schema lock
0926 19:58:49.149549 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:49.149788 (+ 239us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.149813 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1311,"apply.run_cpu_time_us":1630,"apply.run_wall_time_us":1629,"num_ops":2,"prepare.queue_time_us":1081329,"prepare.run_cpu_time_us":2110,"prepare.run_wall_time_us":152707,"raft.queue_time_us":87,"raft.run_cpu_time_us":712,"raft.run_wall_time_us":712,"replication_time_us":174324}]]}
I20260926 19:58:49.121845 12562 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37792 (request call id 179) took 1366 ms. Trace:
I20260926 19:58:49.157033 12562 rpcz_store.cc:276] 0926 19:58:47.755462 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.830835 (+ 75373us) service_pool.cc:224] Handling call
0926 19:58:48.003567 (+172732us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:48.003657 (+ 90us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:48.697985 (+694328us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.117284 (+419299us) tablet_replica.cc:206] Creating consensus instance
0926 19:58:49.121778 (+ 4494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6400}
W20260926 19:58:49.077544 12590 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s user 0.005s sys 0.012s
W20260926 19:58:49.158308 12590 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.678s user 0.007s sys 0.012s
I20260926 19:58:49.162976 12391 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45734 (request call id 250) took 1424 ms. Trace:
I20260926 19:58:49.168028 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 59f7dc92330d400f93f427f3f79ad826 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d153e60ae5d486cb91c2cc623b99985) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:49.172484 13203 ts_tablet_manager.cc:1929] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:49.163537 12391 rpcz_store.cc:276] 0926 19:58:47.738537 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.891111 (+152574us) service_pool.cc:224] Handling call
0926 19:58:47.900416 (+ 9305us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:58:47.902236 (+ 1820us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:58:49.161032 (+1258796us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:58:49.162429 (+ 1397us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:58:49.162882 (+ 453us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:49.123393 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:49.123775 (+ 382us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:49.123799 (+ 24us) write_op.cc:435] Acquired schema lock
0926 19:58:49.123812 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:49.124060 (+ 248us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:49.124095 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:49.124118 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:49.124384 (+ 266us) tablet.cc:701] Row locks acquired
0926 19:58:49.124397 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:49.124507 (+ 110us) write_op.cc:270] Start()
0926 19:58:49.124593 (+ 86us) write_op.cc:276] Timestamp: P: 1790452729124466 usec, L: 0
0926 19:58:49.124614 (+ 21us) op_driver.cc:348] REPLICATION: starting
0926 19:58:49.125237 (+ 623us) log.cc:844] Serialized 433 byte log entry
0926 19:58:49.147821 (+ 22584us) op_driver.cc:464] REPLICATION: finished
0926 19:58:49.150066 (+ 2245us) write_op.cc:301] APPLY: starting
0926 19:58:49.150114 (+ 48us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:49.150219 (+ 105us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:49.150321 (+ 102us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:49.150737 (+ 416us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:49.150877 (+ 140us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:58:49.150923 (+ 46us) write_op.cc:312] APPLY: finished
0926 19:58:49.151087 (+ 164us) log.cc:844] Serialized 29 byte log entry
0926 19:58:49.151223 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.151290 (+ 67us) write_op.cc:454] Released schema lock
0926 19:58:49.151332 (+ 42us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2123,"apply.run_cpu_time_us":1547,"apply.run_wall_time_us":11644,"num_ops":2,"prepare.queue_time_us":1219850,"replication_time_us":23110}]]}
I20260926 19:58:49.190227 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_861452498c1d4bcab2471cd4e97d830a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:49.193460 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_861452498c1d4bcab2471cd4e97d830a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:49.159787 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 59f7dc92330d400f93f427f3f79ad826 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d153e60ae5d486cb91c2cc623b99985) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:49.207446 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 59f7dc92330d400f93f427f3f79ad826 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59f7dc92330d400f93f427f3f79ad826 already in progress: creating tablet
I20260926 19:58:49.209172 13012 ts_tablet_manager.cc:1434] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.346s user 0.004s sys 0.003s
W20260926 19:58:49.214145 12390 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cc9f7a2e5d1c41a8ac793a31696fb311 consensus state has no leader
I20260926 19:58:49.214655 12390 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:45734 (request call id 251) took 1375 ms. Trace:
I20260926 19:58:49.214950 12390 rpcz_store.cc:276] 0926 19:58:47.839028 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:48.095045 (+256017us) service_pool.cc:224] Handling call
0926 19:58:48.100549 (+ 5504us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8be6 00007fb36a68c634 00007fb36a68c5c2 00007fb36a68c4f1 00007fb36a68c46c 00007fb36a68b722 00007fb36892d09c 00007fb36892c4d6
0926 19:58:48.101394 (+ 845us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:48.102374 (+ 980us) catalog_manager.cc:3764] Apply alter schema
0926 19:58:48.103049 (+ 675us) catalog_manager.cc:3796] Acquired catalog manager lock
0926 19:58:48.103296 (+ 247us) catalog_manager.cc:3954] Updating metadata on disk
0926 19:58:49.213641 (+1110345us) catalog_manager.cc:3985] Committing alterations to in-memory state
0926 19:58:49.214598 (+ 957us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:49.194658 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:49.195204 (+ 546us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:49.195241 (+ 37us) write_op.cc:435] Acquired schema lock
0926 19:58:49.195263 (+ 22us) tablet.cc:662] Decoding operations
0926 19:58:49.195601 (+ 338us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:49.195651 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:49.195682 (+ 31us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:49.195854 (+ 172us) tablet.cc:701] Row locks acquired
0926 19:58:49.195872 (+ 18us) write_op.cc:260] PREPARE: finished
0926 19:58:49.195992 (+ 120us) write_op.cc:270] Start()
0926 19:58:49.196099 (+ 107us) write_op.cc:276] Timestamp: P: 1790452729195948 usec, L: 0
0926 19:58:49.196123 (+ 24us) op_driver.cc:348] REPLICATION: starting
0926 19:58:49.196710 (+ 587us) log.cc:844] Serialized 405 byte log entry
0926 19:58:49.206007 (+ 9297us) op_driver.cc:464] REPLICATION: finished
0926 19:58:49.211845 (+ 5838us) write_op.cc:301] APPLY: starting
0926 19:58:49.211907 (+ 62us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:49.212018 (+ 111us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:49.212148 (+ 130us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:49.212325 (+ 177us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:49.212477 (+ 152us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:49.212529 (+ 52us) write_op.cc:312] APPLY: finished
0926 19:58:49.212695 (+ 166us) log.cc:844] Serialized 23 byte log entry
0926 19:58:49.213336 (+ 641us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.213399 (+ 63us) write_op.cc:454] Released schema lock
0926 19:58:49.213430 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":11050752,"child_traces":[["op",{"apply.queue_time_us":5666,"num_ops":1,"prepare.queue_time_us":1087859,"prepare.run_cpu_time_us":2415,"prepare.run_wall_time_us":2415,"raft.queue_time_us":1018,"raft.run_cpu_time_us":784,"raft.run_wall_time_us":11689,"replication_time_us":9776}]]}
I20260926 19:58:49.220428 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_5bf4644212d544c696bffa35c366b0f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:49.222184 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5bf4644212d544c696bffa35c366b0f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:49.224959 13092 tablet_bootstrap.cc:492] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:49.225490 13092 ts_tablet_manager.cc:1403] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.397s user 0.008s sys 0.001s
I20260926 19:58:49.233203 13203 log.cc:1199] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/0fd184e6295d47aeba809be6aab9ce1c
I20260926 19:58:49.225924 13013 tablet_bootstrap.cc:492] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:49.235090 13013 ts_tablet_manager.cc:1403] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.215s user 0.023s sys 0.019s
I20260926 19:58:49.240949 13013 raft_consensus.cc:359] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.241940 13013 raft_consensus.cc:385] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.242304 13013 raft_consensus.cc:740] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.244808 12545 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37786 (request call id 29) took 1108 ms. Trace:
W20260926 19:58:49.251184 12696 consensus_peers.cc:597] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:49.255856 12545 rpcz_store.cc:276] 0926 19:58:48.177582 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:48.179381 (+ 1799us) service_pool.cc:224] Handling call
0926 19:58:48.592682 (+413301us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5400061654. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a258406b8 00007f9a268b198d 00007f9a25001104 00007f9a25001092 00007f9a25000fc1 00007f9a25000f3c 00007f9a250001f2 00007f9a2351509c 00007f9a235144d6 00007f9a23513e7f
0926 19:58:48.688545 (+ 95863us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b5400062f54. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a258406b8 00007f9a268b198d 00007f9a25001104 00007f9a25001092 00007f9a25000fc1 00007f9a25000f3c 00007f9a250001f2 00007f9a2351509c 00007f9a235144d6 00007f9a23513e7f
0926 19:58:48.960143 (+271598us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54000938d4. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a258406b8 00007f9a268b198d 00007f9a25001104 00007f9a25001092 00007f9a25000fc1 00007f9a25000f3c 00007f9a250001f2 00007f9a2351509c 00007f9a235144d6 00007f9a23513e7f
0926 19:58:49.236994 (+276851us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400100754. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a258406b8 00007f9a268b198d 00007f9a25001104 00007f9a25001092 00007f9a25000fc1 00007f9a25000f3c 00007f9a250001f2 00007f9a2351509c 00007f9a235144d6 00007f9a23513e7f
0926 19:58:49.244572 (+ 7578us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387506304}
I20260926 19:58:49.262305 13012 tablet_bootstrap.cc:492] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:49.252723 13509 tablet.cc:1751] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:49.234436 13203 ts_tablet_manager.cc:1950] T 0fd184e6295d47aeba809be6aab9ce1c P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:49.267704 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 166) took 3152 ms. Trace:
I20260926 19:58:49.268162 13203 rpcz_store.cc:276] 0926 19:58:46.114841 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.119003 (+ 4162us) service_pool.cc:224] Handling call
0926 19:58:48.387082 (+2268079us) ts_tablet_manager.cc:1151] Deleting tablet 0fd184e6295d47aeba809be6aab9ce1c
0926 19:58:48.387152 (+ 70us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:48.677299 (+290147us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54000d1e54. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a2584eb22 00007f9a25c8f55c 00007f9a268f8c3a 00007f9a26929960 00007f9a2690fe6d 00007f9a2690fe19 00007f9a2690fda9 00007f9a2690fd71 00007f9a2690ef9d 00000000005bd4e4
0926 19:58:49.047579 (+370280us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.172357 (+124778us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.267621 (+ 95264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583766784,"tablet-delete.queue_time_us":2267448}
I20260926 19:58:49.282128 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 0fd184e6295d47aeba809be6aab9ce1c (table default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]) successfully deleted
I20260926 19:58:49.259927 13013 consensus_queue.cc:260] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.284197 13013 ts_tablet_manager.cc:1434] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.049s user 0.007s sys 0.000s
I20260926 19:58:49.285012 13013 tablet_bootstrap.cc:492] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:49.286818 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 167) took 3068 ms. Trace:
I20260926 19:58:49.289161 13203 rpcz_store.cc:276] 0926 19:58:46.218086 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.218277 (+ 191us) service_pool.cc:224] Handling call
0926 19:58:49.286462 (+3068185us) ts_tablet_manager.cc:1151] Deleting tablet cd86f01d82d041f4852f83f2547e697f
0926 19:58:49.286522 (+ 60us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.286768 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3067636}
W20260926 19:58:49.290349 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet cd86f01d82d041f4852f83f2547e697f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd86f01d82d041f4852f83f2547e697f already in progress: creating tablet
I20260926 19:58:49.298784 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 168) took 2892 ms. Trace:
I20260926 19:58:49.299170 13203 rpcz_store.cc:276] 0926 19:58:46.406819 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.407678 (+ 859us) service_pool.cc:224] Handling call
0926 19:58:49.291122 (+2883444us) ts_tablet_manager.cc:1151] Deleting tablet c5b734f0d33342d3be8b1827bcd5dd14
0926 19:58:49.298443 (+ 7321us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.298729 (+ 286us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2882766}
W20260926 19:58:49.302618 12521 consensus_peers.cc:597] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3215bbd5be56438491a73925ac9ccbe6. This is attempt 1: this message will repeat every 5th retry.
W20260926 19:58:49.306135 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet c5b734f0d33342d3be8b1827bcd5dd14 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5b734f0d33342d3be8b1827bcd5dd14 already in progress: creating tablet
I20260926 19:58:49.307291 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 169) took 2795 ms. Trace:
I20260926 19:58:49.304926 13013 tablet_bootstrap.cc:654] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:49.307655 13203 rpcz_store.cc:276] 0926 19:58:46.511458 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.516814 (+ 5356us) service_pool.cc:224] Handling call
0926 19:58:49.306922 (+2790108us) ts_tablet_manager.cc:1151] Deleting tablet b32f8b0d480547a58ba5c6fed6483a74
0926 19:58:49.306985 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.307239 (+ 254us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2784490}
W20260926 19:58:49.308974 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet b32f8b0d480547a58ba5c6fed6483a74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b32f8b0d480547a58ba5c6fed6483a74 already in progress: creating tablet
I20260926 19:58:49.314426 13203 tablet_replica.cc:333] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:49.315182 13203 raft_consensus.cc:2243] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:49.316131 13203 raft_consensus.cc:2272] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:49.329735 13203 ts_tablet_manager.cc:1916] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:49.229048 13092 raft_consensus.cc:359] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:49.341904 13092 raft_consensus.cc:385] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.342211 13092 raft_consensus.cc:740] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.343230 13092 consensus_queue.cc:260] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:49.370337 13012 tablet_bootstrap.cc:654] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:49.373310 13403 ts_tablet_manager.cc:1929] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:49.373960 13403 log.cc:1199] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/0fd184e6295d47aeba809be6aab9ce1c
I20260926 19:58:49.374761 13403 ts_tablet_manager.cc:1950] T 0fd184e6295d47aeba809be6aab9ce1c P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:49.376940 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 158) took 3115 ms. Trace:
I20260926 19:58:49.379805 13013 tablet_bootstrap.cc:492] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:49.380183 13013 ts_tablet_manager.cc:1403] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.095s user 0.010s sys 0.010s
I20260926 19:58:49.387616 13203 ts_tablet_manager.cc:1929] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:49.387982 13203 log.cc:1199] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/b82d75be43e841e7abcc9e9db800aa2d
I20260926 19:58:49.388631 13203 ts_tablet_manager.cc:1950] T b82d75be43e841e7abcc9e9db800aa2d P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:49.389341 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 170) took 2581 ms. Trace:
I20260926 19:58:49.389650 13203 rpcz_store.cc:276] 0926 19:58:46.808100 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.809471 (+ 1371us) service_pool.cc:224] Handling call
0926 19:58:49.314243 (+2504772us) ts_tablet_manager.cc:1151] Deleting tablet b82d75be43e841e7abcc9e9db800aa2d
0926 19:58:49.314304 (+ 61us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.364365 (+ 50061us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.387503 (+ 23138us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.389294 (+ 1791us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2493495}
I20260926 19:58:49.394138 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet b82d75be43e841e7abcc9e9db800aa2d (table default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa]) successfully deleted
I20260926 19:58:49.395306 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 181) took 1464 ms. Trace:
I20260926 19:58:49.395618 13203 rpcz_store.cc:276] 0926 19:58:47.931230 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.931348 (+ 118us) service_pool.cc:224] Handling call
0926 19:58:49.394959 (+1463611us) ts_tablet_manager.cc:1151] Deleting tablet 8f59be98ceef4effa15baf487c2d1140
0926 19:58:49.394998 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.395238 (+ 240us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1462899}
W20260926 19:58:49.397126 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 8f59be98ceef4effa15baf487c2d1140 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8f59be98ceef4effa15baf487c2d1140 already in progress: creating tablet
I20260926 19:58:49.397874 13013 raft_consensus.cc:359] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.398512 13013 raft_consensus.cc:385] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.398782 13013 raft_consensus.cc:740] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.413686 13013 consensus_queue.cc:260] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.420902 13409 raft_consensus.cc:493] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:49.421372 13409 raft_consensus.cc:515] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.422781 13012 tablet_bootstrap.cc:492] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:49.423166 13012 ts_tablet_manager.cc:1403] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.193s user 0.009s sys 0.001s
I20260926 19:58:49.423744 13409 leader_election.cc:290] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:49.382110 13498 raft_consensus.cc:493] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:49.426811 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa82c030ec34ac5a8c448d80cdbae95" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:49.427191 13498 raft_consensus.cc:515] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:58:49.428125 12696 leader_election.cc:343] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:49.424865 12590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa82c030ec34ac5a8c448d80cdbae95" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:49.429849 12590 raft_consensus.cc:2468] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:49.431059 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7d5e580f5645ab95fa03d5f669dc32" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:49.431493 13013 ts_tablet_manager.cc:1434] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.051s user 0.008s sys 0.000s
I20260926 19:58:49.431252 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7d5e580f5645ab95fa03d5f669dc32" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:49.432915 12858 leader_election.cc:304] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:49.433669 13514 raft_consensus.cc:2749] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:49.434361 13013 tablet_bootstrap.cc:492] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:49.429199 13498 leader_election.cc:290] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:49.451280 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 182) took 1519 ms. Trace:
I20260926 19:58:49.452216 13464 raft_consensus.cc:493] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:49.452665 13464 raft_consensus.cc:515] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.451644 13203 rpcz_store.cc:276] 0926 19:58:47.931558 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.932137 (+ 579us) service_pool.cc:224] Handling call
0926 19:58:49.450167 (+1518030us) ts_tablet_manager.cc:1151] Deleting tablet d48ee26c0aaa4abc8e08a1a2dadb028d
0926 19:58:49.450217 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.450493 (+ 276us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1517518}
I20260926 19:58:49.455046 13464 leader_election.cc:290] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:49.399421 13092 ts_tablet_manager.cc:1434] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.174s user 0.007s sys 0.000s
I20260926 19:58:49.455060 13203 tablet_replica.cc:333] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:49.457613 12695 leader_election.cc:304] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:49.457710 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b7715b8fcb4b59917ae14a83f9955d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:49.404661 12924 raft_consensus.cc:3060] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:49.458415 13409 raft_consensus.cc:2804] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260926 19:58:49.458989 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet d48ee26c0aaa4abc8e08a1a2dadb028d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d48ee26c0aaa4abc8e08a1a2dadb028d already in progress: creating tablet
I20260926 19:58:49.458923 13409 raft_consensus.cc:493] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:49.459367 13409 raft_consensus.cc:3060] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:49.459741 12519 leader_election.cc:343] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:49.456446 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b7715b8fcb4b59917ae14a83f9955d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:49.462888 12521 leader_election.cc:343] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:49.463232 12521 leader_election.cc:304] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:49.464362 13464 raft_consensus.cc:2749] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:49.471442 13013 tablet_bootstrap.cc:654] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:49.473064 13092 tablet_bootstrap.cc:492] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:49.487007 13403 rpcz_store.cc:276] 0926 19:58:46.261390 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.261530 (+ 140us) service_pool.cc:224] Handling call
0926 19:58:48.783319 (+2521789us) ts_tablet_manager.cc:1151] Deleting tablet 0fd184e6295d47aeba809be6aab9ce1c
0926 19:58:48.783370 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.120454 (+337084us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.373180 (+252726us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.376880 (+ 3700us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2521343}
I20260926 19:58:49.491161 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 159) took 3014 ms. Trace:
I20260926 19:58:49.491732 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 0fd184e6295d47aeba809be6aab9ce1c (table default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]) successfully deleted
I20260926 19:58:49.494015 13203 raft_consensus.cc:2243] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:49.494906 13203 raft_consensus.cc:2272] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:49.503484 13403 rpcz_store.cc:276] 0926 19:58:46.476366 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.476495 (+ 129us) service_pool.cc:224] Handling call
0926 19:58:49.490748 (+3014253us) ts_tablet_manager.cc:1151] Deleting tablet c5b734f0d33342d3be8b1827bcd5dd14
0926 19:58:49.490811 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.491102 (+ 291us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3013712}
W20260926 19:58:49.504952 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet c5b734f0d33342d3be8b1827bcd5dd14 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5b734f0d33342d3be8b1827bcd5dd14 already in progress: creating tablet
I20260926 19:58:49.512584 13203 ts_tablet_manager.cc:1916] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:49.508849 13409 raft_consensus.cc:515] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:49.518481 12395 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0a7d5e580f5645ab95fa03d5f669dc32 consensus state has no leader
I20260926 19:58:49.528368 13498 consensus_queue.cc:1048] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:49.556434 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa82c030ec34ac5a8c448d80cdbae95" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:49.558573 12696 leader_election.cc:343] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:49.560063 13409 leader_election.cc:290] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:49.561050 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa82c030ec34ac5a8c448d80cdbae95" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:49.561643 12592 raft_consensus.cc:3060] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:49.511992 13012 raft_consensus.cc:359] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.562106 13012 raft_consensus.cc:385] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.563462 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 160) took 3058 ms. Trace:
I20260926 19:58:49.564181 13012 raft_consensus.cc:740] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.567265 13012 consensus_queue.cc:260] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.568126 13403 rpcz_store.cc:276] 0926 19:58:46.505271 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.505376 (+ 105us) service_pool.cc:224] Handling call
0926 19:58:49.524774 (+3019398us) ts_tablet_manager.cc:1151] Deleting tablet b32f8b0d480547a58ba5c6fed6483a74
0926 19:58:49.563134 (+ 38360us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.563405 (+ 271us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3018868}
I20260926 19:58:49.568919 12592 raft_consensus.cc:2468] T 1aa82c030ec34ac5a8c448d80cdbae95 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:49.500974 13092 tablet_bootstrap.cc:654] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:49.570962 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet b32f8b0d480547a58ba5c6fed6483a74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b32f8b0d480547a58ba5c6fed6483a74 already in progress: creating tablet
I20260926 19:58:49.574055 13013 tablet_bootstrap.cc:492] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:49.575496 13403 tablet_replica.cc:333] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:49.570183 12695 leader_election.cc:304] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:49.577138 13432 raft_consensus.cc:2804] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:49.577581 13432 raft_consensus.cc:697] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:49.578317 13432 consensus_queue.cc:237] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.578883 13012 ts_tablet_manager.cc:1434] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.155s user 0.006s sys 0.001s
I20260926 19:58:49.607890 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_010ba86b94b74ad18873d9aefd9c2849"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:49.610834 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_010ba86b94b74ad18873d9aefd9c2849 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:49.616784 13541 raft_consensus.cc:493] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:49.617368 13541 raft_consensus.cc:515] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.619275 13541 leader_election.cc:290] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:49.622603 13013 ts_tablet_manager.cc:1403] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.190s user 0.012s sys 0.012s
I20260926 19:58:49.627028 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b7715b8fcb4b59917ae14a83f9955d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:49.630774 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b7715b8fcb4b59917ae14a83f9955d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:49.631399 12592 raft_consensus.cc:2468] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:49.582815 13403 raft_consensus.cc:2243] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:49.632766 13403 raft_consensus.cc:2272] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:49.634737 13403 ts_tablet_manager.cc:1916] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:49.652550 13013 raft_consensus.cc:359] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.653190 13013 raft_consensus.cc:385] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.653450 13013 raft_consensus.cc:740] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:49.655848 12381 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (table default.Table_d6b47ef12b28492bb711ca846f80f6cb [id=c2c079dc87ec480a82aa5b57b110da17], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0a7d5e580f5645ab95fa03d5f669dc32 consensus state has no leader
I20260926 19:58:49.670217 13092 tablet_bootstrap.cc:492] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:49.670665 13092 ts_tablet_manager.cc:1403] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.198s user 0.012s sys 0.002s
I20260926 19:58:49.638609 13012 tablet_bootstrap.cc:492] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:49.672353 13013 consensus_queue.cc:260] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.674602 13543 consensus_queue.cc:1048] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:49.681154 13012 tablet_bootstrap.cc:654] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:49.690037 13092 raft_consensus.cc:359] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.690948 13092 raft_consensus.cc:385] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.691303 13092 raft_consensus.cc:740] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.692157 13092 consensus_queue.cc:260] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.692881 13013 ts_tablet_manager.cc:1434] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.070s user 0.005s sys 0.000s
I20260926 19:58:49.693873 13013 tablet_bootstrap.cc:492] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:49.650929 12695 leader_election.cc:304] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
W20260926 19:58:49.709878 12696 leader_election.cc:343] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:49.714915 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 64067.2us
I20260926 19:58:49.715291 13464 consensus_queue.cc:1048] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:49.716522 13542 raft_consensus.cc:2804] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:49.717058 13542 raft_consensus.cc:493] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:49.717550 13542 raft_consensus.cc:3060] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:49.728359 13092 ts_tablet_manager.cc:1434] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.047s user 0.005s sys 0.000s
I20260926 19:58:49.742009 13012 tablet_bootstrap.cc:492] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:49.742707 13012 ts_tablet_manager.cc:1403] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.104s user 0.012s sys 0.000s
I20260926 19:58:49.753130 13092 tablet_bootstrap.cc:492] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:49.772970 12390 master_service.cc:960] ReplaceTablet: received request to replace tablet 0fd184e6295d47aeba809be6aab9ce1c from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:49.774914 13012 raft_consensus.cc:359] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.782580 13012 raft_consensus.cc:385] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.784566 13012 raft_consensus.cc:740] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
W20260926 19:58:49.790287 12521 consensus_peers.cc:597] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:49.796505 13012 consensus_queue.cc:260] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.814226 13403 ts_tablet_manager.cc:1929] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:49.814733 13403 log.cc:1199] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/b82d75be43e841e7abcc9e9db800aa2d
I20260926 19:58:49.815640 13403 ts_tablet_manager.cc:1950] T b82d75be43e841e7abcc9e9db800aa2d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:49.817349 13012 ts_tablet_manager.cc:1434] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.074s user 0.004s sys 0.002s
I20260926 19:58:49.827171 13012 tablet_bootstrap.cc:492] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:49.829735 13013 tablet_bootstrap.cc:654] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:49.846246 12696 consensus_peers.cc:597] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fd184e6295d47aeba809be6aab9ce1c. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:49.844539 13203 ts_tablet_manager.cc:1929] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:49.846887 13203 log.cc:1199] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/9acbd4e5aa23433d860d399d78485d0d
W20260926 19:58:49.847613 12695 consensus_peers.cc:597] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fd184e6295d47aeba809be6aab9ce1c. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:49.847957 13203 ts_tablet_manager.cc:1950] T 9acbd4e5aa23433d860d399d78485d0d P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:49.857033 13554 raft_consensus.cc:493] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:49.857975 13554 raft_consensus.cc:515] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.871601 13554 leader_election.cc:290] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:49.855232 13012 tablet_bootstrap.cc:654] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:49.852568 13542 raft_consensus.cc:515] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.870826 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16df0828847d4e7db5702d84082fa402" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:49.879127 12519 leader_election.cc:343] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:49.880620 13542 leader_election.cc:290] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:49.871031 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 183) took 1655 ms. Trace:
I20260926 19:58:49.882134 13203 rpcz_store.cc:276] 0926 19:58:48.215982 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:48.216766 (+ 784us) service_pool.cc:224] Handling call
0926 19:58:49.454915 (+1238149us) ts_tablet_manager.cc:1151] Deleting tablet 9acbd4e5aa23433d860d399d78485d0d
0926 19:58:49.454957 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.621178 (+166221us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.844337 (+223159us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.870937 (+ 26600us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1237537}
I20260926 19:58:49.884050 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 9acbd4e5aa23433d860d399d78485d0d (table default.Table_e0ad88ad494c42759968cb1dc8501282 [id=6890560cf0104034a9eb9431a38c99ec]) successfully deleted
I20260926 19:58:49.884861 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 161) took 3079 ms. Trace:
I20260926 19:58:49.833560 13092 tablet_bootstrap.cc:654] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:49.861233 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16df0828847d4e7db5702d84082fa402" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:49.872292 12858 consensus_peers.cc:597] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd92a1a62ab24647bef7c9eefdf9c9d8. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:49.882516 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b7715b8fcb4b59917ae14a83f9955d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:49.887745 12696 leader_election.cc:343] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:49.888706 12521 leader_election.cc:343] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:49.889258 12521 leader_election.cc:304] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:49.893910 13554 raft_consensus.cc:2749] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:49.896119 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b7715b8fcb4b59917ae14a83f9955d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:49.896782 12592 raft_consensus.cc:3060] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:49.898926 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 59f7dc92330d400f93f427f3f79ad826 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59f7dc92330d400f93f427f3f79ad826 already in progress: creating tablet
I20260926 19:58:49.917062 12592 raft_consensus.cc:2468] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:49.918862 12695 leader_election.cc:304] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:49.920729 13541 raft_consensus.cc:2804] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:49.921382 13541 raft_consensus.cc:697] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:49.922982 13541 consensus_queue.cc:237] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.885185 13403 rpcz_store.cc:276] 0926 19:58:46.805409 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:46.805640 (+ 231us) service_pool.cc:224] Handling call
0926 19:58:49.575338 (+2769698us) ts_tablet_manager.cc:1151] Deleting tablet b82d75be43e841e7abcc9e9db800aa2d
0926 19:58:49.575383 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.719613 (+144230us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.814083 (+ 94470us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:49.884798 (+ 70715us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2768971}
I20260926 19:58:49.940315 13013 tablet_bootstrap.cc:492] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:49.940794 13013 ts_tablet_manager.cc:1403] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.247s user 0.015s sys 0.010s
I20260926 19:58:49.943325 13013 raft_consensus.cc:359] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.944020 13013 raft_consensus.cc:385] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.944311 13013 raft_consensus.cc:740] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.945221 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet b82d75be43e841e7abcc9e9db800aa2d (table default.Table_fe617b43605c467aa4ded460c57b4624 [id=22559b05bf1541c09bbc6ae9689bb8fa]) successfully deleted
I20260926 19:58:49.947430 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 171) took 2076 ms. Trace:
I20260926 19:58:49.950397 13403 rpcz_store.cc:276] 0926 19:58:47.870714 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.870839 (+ 125us) service_pool.cc:224] Handling call
0926 19:58:49.946928 (+2076089us) ts_tablet_manager.cc:1151] Deleting tablet 8f59be98ceef4effa15baf487c2d1140
0926 19:58:49.946991 (+ 63us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.947352 (+ 361us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2075595}
I20260926 19:58:49.954401 13012 tablet_bootstrap.cc:492] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:49.954834 13012 ts_tablet_manager.cc:1403] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.128s user 0.008s sys 0.011s
I20260926 19:58:49.960763 13092 tablet_bootstrap.cc:492] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:49.965343 13092 ts_tablet_manager.cc:1403] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.212s user 0.013s sys 0.001s
W20260926 19:58:49.964319 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 8f59be98ceef4effa15baf487c2d1140 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8f59be98ceef4effa15baf487c2d1140 already in progress: creating tablet
I20260926 19:58:49.968001 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 172) took 2092 ms. Trace:
I20260926 19:58:49.975401 13403 rpcz_store.cc:276] 0926 19:58:47.875790 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.885429 (+ 9639us) service_pool.cc:224] Handling call
0926 19:58:49.967619 (+2082190us) ts_tablet_manager.cc:1151] Deleting tablet d48ee26c0aaa4abc8e08a1a2dadb028d
0926 19:58:49.967664 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:49.967941 (+ 277us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2080907}
I20260926 19:58:49.957353 13012 raft_consensus.cc:359] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.976895 13012 raft_consensus.cc:385] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:49.977169 13012 raft_consensus.cc:740] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.978894 13013 consensus_queue.cc:260] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.982323 13012 consensus_queue.cc:260] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.991544 13403 tablet_replica.cc:333] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:49.985460 13013 ts_tablet_manager.cc:1434] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.044s user 0.006s sys 0.001s
I20260926 19:58:49.974716 13092 raft_consensus.cc:359] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:49.998999 13012 ts_tablet_manager.cc:1434] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.044s user 0.005s sys 0.002s
I20260926 19:58:49.999974 13012 tablet_bootstrap.cc:492] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:50.002108 13013 tablet_bootstrap.cc:492] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:49.998478 13092 raft_consensus.cc:385] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:50.006230 13092 raft_consensus.cc:740] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:49.997843 13403 raft_consensus.cc:2243] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:50.008078 13403 raft_consensus.cc:2272] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:50.013221 13403 ts_tablet_manager.cc:1916] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:50.017285 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet d48ee26c0aaa4abc8e08a1a2dadb028d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d48ee26c0aaa4abc8e08a1a2dadb028d already in progress: creating tablet
I20260926 19:58:50.029896 13013 tablet_bootstrap.cc:654] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:50.018738 13543 consensus_queue.cc:1048] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:50.037714 12900 tablet_service.cc:1511] Processing CreateTablet for tablet e1b4aff22b9e4932beff6c2b90a9ab1e (DEFAULT_TABLE table=default.Table_2c9b200af3d54549b7c8184d0b0a3d44 [id=1e63508a8a7a4445b2d36f273cce75a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.038757 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1b4aff22b9e4932beff6c2b90a9ab1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.038666 12739 tablet_service.cc:1511] Processing CreateTablet for tablet e1b4aff22b9e4932beff6c2b90a9ab1e (DEFAULT_TABLE table=default.Table_2c9b200af3d54549b7c8184d0b0a3d44 [id=1e63508a8a7a4445b2d36f273cce75a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.039664 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1b4aff22b9e4932beff6c2b90a9ab1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.041409 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 7df6f68f3590442fb2d1a1f7e2df9d7e (DEFAULT_TABLE table=default.Table_f7366985fc9a4e2386af9d2847ab0424 [id=cfadc44e164b4d7d9f4068ed36f945cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.042312 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7df6f68f3590442fb2d1a1f7e2df9d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.042197 12562 tablet_service.cc:1511] Processing CreateTablet for tablet e1b4aff22b9e4932beff6c2b90a9ab1e (DEFAULT_TABLE table=default.Table_2c9b200af3d54549b7c8184d0b0a3d44 [id=1e63508a8a7a4445b2d36f273cce75a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.042400 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 7df6f68f3590442fb2d1a1f7e2df9d7e (DEFAULT_TABLE table=default.Table_f7366985fc9a4e2386af9d2847ab0424 [id=cfadc44e164b4d7d9f4068ed36f945cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.045612 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 6e20b1a22af944a0aad0fee5fc835f6e (DEFAULT_TABLE table=default.Table_0e1b1c5a52dc4176b9b867aa57c3ff1c [id=a07ef56319a44e10a30f55432289b933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.048723 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 7df6f68f3590442fb2d1a1f7e2df9d7e (DEFAULT_TABLE table=default.Table_f7366985fc9a4e2386af9d2847ab0424 [id=cfadc44e164b4d7d9f4068ed36f945cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.050449 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 6e20b1a22af944a0aad0fee5fc835f6e (DEFAULT_TABLE table=default.Table_0e1b1c5a52dc4176b9b867aa57c3ff1c [id=a07ef56319a44e10a30f55432289b933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.050582 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 13bb4e7eeef145fc85a5c412add99ce7 (DEFAULT_TABLE table=default.Table_019731a69622436792bb7a8512e0cb49 [id=f2cc65dc707940e5a763c0d4b768a072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.051472 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e20b1a22af944a0aad0fee5fc835f6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.048063 12387 catalog_manager.cc:5838] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195). New cstate: current_term: 1 leader_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:50.053421 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:50.054162 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]). Expected version 2 got 0
I20260926 19:58:50.058573 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 13bb4e7eeef145fc85a5c412add99ce7 (DEFAULT_TABLE table=default.Table_019731a69622436792bb7a8512e0cb49 [id=f2cc65dc707940e5a763c0d4b768a072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.059600 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13bb4e7eeef145fc85a5c412add99ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.062875 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7df6f68f3590442fb2d1a1f7e2df9d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.069777 13554 consensus_queue.cc:1048] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:50.072103 12521 consensus_peers.cc:597] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:50.079207 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 8fe5c9795b7b41e4815f2c097c494cd2 (DEFAULT_TABLE table=default.Table_498bd18aa31b447993f6ed4f665070a5 [id=3e2b31ebeb554297b7e44bf5fd8e17e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.080158 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fe5c9795b7b41e4815f2c097c494cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.080103 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_15cc720c12e6419d8ca44d8485d7c80b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:50.081540 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15cc720c12e6419d8ca44d8485d7c80b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:50.086943 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 6e20b1a22af944a0aad0fee5fc835f6e (DEFAULT_TABLE table=default.Table_0e1b1c5a52dc4176b9b867aa57c3ff1c [id=a07ef56319a44e10a30f55432289b933]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.087534 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 13bb4e7eeef145fc85a5c412add99ce7 (DEFAULT_TABLE table=default.Table_019731a69622436792bb7a8512e0cb49 [id=f2cc65dc707940e5a763c0d4b768a072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.088068 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e20b1a22af944a0aad0fee5fc835f6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.088658 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13bb4e7eeef145fc85a5c412add99ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.102977 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1b4aff22b9e4932beff6c2b90a9ab1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.103577 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13bb4e7eeef145fc85a5c412add99ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.106904 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 8fe5c9795b7b41e4815f2c097c494cd2 (DEFAULT_TABLE table=default.Table_498bd18aa31b447993f6ed4f665070a5 [id=3e2b31ebeb554297b7e44bf5fd8e17e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.107882 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fe5c9795b7b41e4815f2c097c494cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.109750 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e20b1a22af944a0aad0fee5fc835f6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.110920 12924 raft_consensus.cc:1275] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 2 FOLLOWER]: Refusing update from remote peer f6e9d6439fd54b28925a68a6145d3dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260926 19:58:50.110080 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f1a85473b0594a75947fbf9a21bd5469" } new_table_name: "default.Table_1dfed69d00a647d1814ee455d0d3b5e2" modify_external_catalogs: true
I20260926 19:58:50.113735 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7df6f68f3590442fb2d1a1f7e2df9d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.118553 13554 consensus_queue.cc:1048] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:50.121768 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 8fe5c9795b7b41e4815f2c097c494cd2 (DEFAULT_TABLE table=default.Table_498bd18aa31b447993f6ed4f665070a5 [id=3e2b31ebeb554297b7e44bf5fd8e17e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.122648 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fe5c9795b7b41e4815f2c097c494cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.008713 13092 consensus_queue.cc:260] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.129418 13092 ts_tablet_manager.cc:1434] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.163s user 0.005s sys 0.000s
I20260926 19:58:50.136848 13543 consensus_queue.cc:1048] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260926 19:58:50.142843 12858 consensus_peers.cc:597] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1455b99a65ff44ff933825196708f485. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:50.145738 13092 tablet_bootstrap.cc:492] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:50.159420 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]). Expected version 2 got 0
I20260926 19:58:50.160481 12395 catalog_manager.cc:5838] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:50.162832 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (table default.Table_d6b47ef12b28492bb711ca846f80f6cb [id=c2c079dc87ec480a82aa5b57b110da17]). Expected version 3 got 0
I20260926 19:58:50.163935 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:50.164878 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 22f268b4a1f7423f917731c6b62aab3e
I20260926 19:58:50.166306 12394 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:45734 (request call id 253) took 1848 ms. Trace:
I20260926 19:58:50.166471 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f268b4a1f7423f917731c6b62aab3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:50.167987 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 22f268b4a1f7423f917731c6b62aab3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22f268b4a1f7423f917731c6b62aab3e already in progress: creating tablet
I20260926 19:58:50.166599 12394 rpcz_store.cc:276] 0926 19:58:48.318268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:48.988010 (+669742us) service_pool.cc:224] Handling call
0926 19:58:49.129641 (+141631us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8876 00007fb36a687b34 00007fb36a687ac2 00007fb36a6879f1 00007fb36a68796c 00007fb36a686c22 00007fb36892d09c 00007fb36892c4d6
0926 19:58:49.144405 (+ 14764us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:49.145002 (+ 597us) catalog_manager.cc:2854] Modifying in-memory table state
0926 19:58:49.145112 (+ 110us) catalog_manager.cc:2863] Locking tablets
0926 19:58:49.150082 (+ 4970us) catalog_manager.cc:2877] Removing table and tablets from system table
0926 19:58:49.487303 (+337221us) catalog_manager.cc:2895] Removing table from by-name map
0926 19:58:49.487534 (+ 231us) catalog_manager.cc:2911] Committing in-memory state
0926 19:58:50.164779 (+677245us) catalog_manager.cc:2915] Aborting table tasks
0926 19:58:50.166254 (+ 1475us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:49.313687 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:49.314151 (+ 464us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:49.314179 (+ 28us) write_op.cc:435] Acquired schema lock
0926 19:58:49.314195 (+ 16us) tablet.cc:662] Decoding operations
0926 19:58:49.314528 (+ 333us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:49.314572 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:49.314603 (+ 31us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:49.314924 (+ 321us) tablet.cc:701] Row locks acquired
0926 19:58:49.314941 (+ 17us) write_op.cc:260] PREPARE: finished
0926 19:58:49.315049 (+ 108us) write_op.cc:270] Start()
0926 19:58:49.315138 (+ 89us) write_op.cc:276] Timestamp: P: 1790452729315000 usec, L: 0
0926 19:58:49.315156 (+ 18us) op_driver.cc:348] REPLICATION: starting
0926 19:58:49.315646 (+ 490us) log.cc:844] Serialized 734 byte log entry
0926 19:58:49.485196 (+169550us) op_driver.cc:464] REPLICATION: finished
0926 19:58:49.485989 (+ 793us) write_op.cc:301] APPLY: starting
0926 19:58:49.486037 (+ 48us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:49.486132 (+ 95us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:49.486222 (+ 90us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:49.486459 (+ 237us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:49.486582 (+ 123us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:58:49.486620 (+ 38us) write_op.cc:312] APPLY: finished
0926 19:58:49.486749 (+ 129us) log.cc:844] Serialized 29 byte log entry
0926 19:58:49.486943 (+ 194us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.486998 (+ 55us) write_op.cc:454] Released schema lock
0926 19:58:49.487025 (+ 27us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":323352320,"child_traces":[["op",{"apply.queue_time_us":674,"apply.run_cpu_time_us":1193,"apply.run_wall_time_us":1193,"num_ops":2,"prepare.queue_time_us":161831,"prepare.run_cpu_time_us":2169,"prepare.run_wall_time_us":2168,"replication_time_us":169966,"spinlock_wait_cycles":89472,"thread_start_us":24591,"threads_started":1}]]}
I20260926 19:58:50.170960 12394 master_service.cc:960] ReplaceTablet: received request to replace tablet f2663906e11549e3aa9ae78d88e645ea from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:50.174381 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f268b4a1f7423f917731c6b62aab3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:50.177460 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f268b4a1f7423f917731c6b62aab3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:50.197872 12924 raft_consensus.cc:3060] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:50.199891 13543 raft_consensus.cc:493] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:50.205863 13543 raft_consensus.cc:515] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.208484 13543 leader_election.cc:290] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:50.209239 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dabe03472b640eca7d88e4c54cc2782" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:50.219827 12758 raft_consensus.cc:1275] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 2 FOLLOWER]: Refusing update from remote peer f6e9d6439fd54b28925a68a6145d3dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260926 19:58:50.220300 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 3488f0076b0f4ef19f7cf23a93cd20e3 (DEFAULT_TABLE table=default.Table_87cff927dd5d4f0d9d947b7a5629728c [id=6a77f8956ec6491d915b40237d54a548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.221263 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3488f0076b0f4ef19f7cf23a93cd20e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.221951 13464 consensus_queue.cc:1048] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260926 19:58:50.237807 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dabe03472b640eca7d88e4c54cc2782" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:50.250168 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 3488f0076b0f4ef19f7cf23a93cd20e3 (DEFAULT_TABLE table=default.Table_87cff927dd5d4f0d9d947b7a5629728c [id=6a77f8956ec6491d915b40237d54a548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.250943 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 3488f0076b0f4ef19f7cf23a93cd20e3 (DEFAULT_TABLE table=default.Table_87cff927dd5d4f0d9d947b7a5629728c [id=6a77f8956ec6491d915b40237d54a548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.251268 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3488f0076b0f4ef19f7cf23a93cd20e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.251873 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3488f0076b0f4ef19f7cf23a93cd20e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.254433 12897 tablet_service.cc:1511] Processing CreateTablet for tablet 39ba14bcfe4b406d8051b4397287ca6c (DEFAULT_TABLE table=default.Table_df083bf280b942dfab05341b05afadc6 [id=caed1efebca940858910c41d1e9878f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.255482 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ba14bcfe4b406d8051b4397287ca6c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:50.276983 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 22f268b4a1f7423f917731c6b62aab3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22f268b4a1f7423f917731c6b62aab3e already in progress: creating tablet
I20260926 19:58:50.293814 13012 tablet_bootstrap.cc:700] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent opening tablet: real 0.273s user 0.003s sys 0.000s
I20260926 19:58:50.294514 13012 tablet_bootstrap.cc:654] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:50.297789 12857 leader_election.cc:304] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:50.297969 13554 raft_consensus.cc:493] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:50.298383 13554 raft_consensus.cc:515] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.299312 13514 raft_consensus.cc:2749] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:50.302795 13554 leader_election.cc:290] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:50.311000 12390 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3dabe03472b640eca7d88e4c54cc2782 consensus state has no leader
I20260926 19:58:50.337174 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:50.338927 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:50.340092 12521 leader_election.cc:343] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:50.343389 12519 leader_election.cc:343] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:50.343830 12519 leader_election.cc:304] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:50.345588 13464 raft_consensus.cc:2749] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:50.363458 13013 tablet_bootstrap.cc:492] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:50.364053 13013 ts_tablet_manager.cc:1403] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.362s user 0.006s sys 0.004s
I20260926 19:58:50.367004 13013 raft_consensus.cc:359] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.371490 13542 raft_consensus.cc:493] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:50.371937 13542 raft_consensus.cc:515] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.372826 13092 tablet_bootstrap.cc:654] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:50.374370 13542 leader_election.cc:290] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:50.384851 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d11cb23144691809c554c125cad0e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:50.386360 12696 leader_election.cc:343] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:50.383798 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 39ba14bcfe4b406d8051b4397287ca6c (DEFAULT_TABLE table=default.Table_df083bf280b942dfab05341b05afadc6 [id=caed1efebca940858910c41d1e9878f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.387890 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ba14bcfe4b406d8051b4397287ca6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.399632 12391 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 1 got 0
I20260926 19:58:50.400311 12391 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (table default.Table_d6b47ef12b28492bb711ca846f80f6cb [id=c2c079dc87ec480a82aa5b57b110da17]). Expected version 3 got 0
I20260926 19:58:50.401646 12393 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_13a9930424e74b249d88b4f42195f0ad [id=dd5c0a6b728948278a5dd48f8ec0bf6f] alter complete (version 2)
I20260926 19:58:50.403321 12741 tablet_service.cc:1511] Processing CreateTablet for tablet a26e6b837f67445c9383599b7e52de7b (DEFAULT_TABLE table=default.Table_87fb2b97ed26421ba052e4bbb187763c [id=765c307ad69a4fb2a6befc0c55508351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.404579 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a26e6b837f67445c9383599b7e52de7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.418208 12898 tablet_service.cc:1511] Processing CreateTablet for tablet a26e6b837f67445c9383599b7e52de7b (DEFAULT_TABLE table=default.Table_87fb2b97ed26421ba052e4bbb187763c [id=765c307ad69a4fb2a6befc0c55508351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.422681 13012 tablet_bootstrap.cc:492] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:50.423148 13012 ts_tablet_manager.cc:1403] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.423s user 0.008s sys 0.001s
I20260926 19:58:50.431501 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a26e6b837f67445c9383599b7e52de7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.432024 13012 raft_consensus.cc:359] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.432677 13012 raft_consensus.cc:385] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:50.432932 13012 raft_consensus.cc:740] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:50.425217 13543 raft_consensus.cc:493] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:50.439249 13543 raft_consensus.cc:515] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.442131 13543 leader_election.cc:290] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:50.442975 13012 consensus_queue.cc:260] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.444926 13012 ts_tablet_manager.cc:1434] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260926 19:58:50.445832 13012 tablet_bootstrap.cc:492] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:50.451876 13585 tablet.cc:1751] T 189eaca5fb2b4b0d83a2873e0495e5ff P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:50.454099 13012 tablet_bootstrap.cc:654] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:50.455602 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet f2663906e11549e3aa9ae78d88e645ea
I20260926 19:58:50.457060 12394 catalog_manager.cc:6934] ReplaceTablet: tablet f2663906e11549e3aa9ae78d88e645ea deleted and replaced by tablet d4434cb08d0f41fd96283de49c53b3e8
I20260926 19:58:50.457361 12394 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:45734 (request call id 261) took 1001 ms. Trace:
I20260926 19:58:50.459458 12736 tablet_service.cc:1558] Processing DeleteTablet for tablet f2663906e11549e3aa9ae78d88e645ea with delete_type TABLET_DATA_DELETED (replaced by tablet d4434cb08d0f41fd96283de49c53b3e8) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:50.458221 12394 rpcz_store.cc:276] 0926 19:58:49.455592 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.170862 (+715270us) service_pool.cc:224] Handling call
0926 19:58:50.457311 (+286449us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:50.357666 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:50.358028 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:50.358050 (+ 22us) write_op.cc:435] Acquired schema lock
0926 19:58:50.358062 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:50.360852 (+ 2790us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:50.360888 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:50.360911 (+ 23us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:50.361100 (+ 189us) tablet.cc:701] Row locks acquired
0926 19:58:50.361114 (+ 14us) write_op.cc:260] PREPARE: finished
0926 19:58:50.361216 (+ 102us) write_op.cc:270] Start()
0926 19:58:50.361292 (+ 76us) write_op.cc:276] Timestamp: P: 1790452730361159 usec, L: 0
0926 19:58:50.361307 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:50.369909 (+ 8602us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b5400030d20. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36abdba50 00007fb36ac65302 00007fb36ac5a727 00007fb36ac5a564 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11
0926 19:58:50.370374 (+ 465us) log.cc:844] Serialized 631 byte log entry
0926 19:58:50.452540 (+ 82166us) op_driver.cc:464] REPLICATION: finished
0926 19:58:50.452775 (+ 235us) write_op.cc:301] APPLY: starting
0926 19:58:50.452827 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:50.452930 (+ 103us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:50.452951 (+ 21us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:50.453265 (+ 314us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:50.453463 (+ 198us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:50.453928 (+ 465us) write_op.cc:312] APPLY: finished
0926 19:58:50.454132 (+ 204us) log.cc:844] Serialized 29 byte log entry
0926 19:58:50.454268 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.454339 (+ 71us) write_op.cc:454] Released schema lock
0926 19:58:50.454375 (+ 36us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":86,"apply.run_cpu_time_us":1414,"apply.run_wall_time_us":1801,"mutex_wait_us":152355,"num_ops":2,"prepare.queue_time_us":183804,"raft.queue_time_us":22265,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":271,"replication_time_us":91151,"spinlock_wait_cycles":17570560,"thread_start_us":79960,"threads_started":1}]]}
W20260926 19:58:50.461270 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet f2663906e11549e3aa9ae78d88e645ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2663906e11549e3aa9ae78d88e645ea already in progress: creating tablet
I20260926 19:58:50.471480 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet f2663906e11549e3aa9ae78d88e645ea with delete_type TABLET_DATA_DELETED (replaced by tablet d4434cb08d0f41fd96283de49c53b3e8) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:50.486438 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 0df471a7b6554d4fb5007adb63babe43 (DEFAULT_TABLE table=default.Table_935be674b5a9419296be4ffa3464b082 [id=d2972f37229845e396114c265b371572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.487699 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df471a7b6554d4fb5007adb63babe43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.492053 13590 tablet.cc:1751] T 189eaca5fb2b4b0d83a2873e0495e5ff P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260926 19:58:50.499126 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 59362.8us
I20260926 19:58:50.500631 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 0df471a7b6554d4fb5007adb63babe43 (DEFAULT_TABLE table=default.Table_935be674b5a9419296be4ffa3464b082 [id=d2972f37229845e396114c265b371572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.500995 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:50.501611 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df471a7b6554d4fb5007adb63babe43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.502074 12756 raft_consensus.cc:2393] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:50.508355 13587 consensus_queue.cc:1048] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:50.523979 13541 raft_consensus.cc:493] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:50.524753 13541 raft_consensus.cc:515] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.527696 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ef773d1c6694c63a011819024451b02" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:50.528944 12696 leader_election.cc:343] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:50.378456 12590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d11cb23144691809c554c125cad0e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:50.409276 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 39ba14bcfe4b406d8051b4397287ca6c (DEFAULT_TABLE table=default.Table_df083bf280b942dfab05341b05afadc6 [id=caed1efebca940858910c41d1e9878f3]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:50.427273 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 83919.3us
I20260926 19:58:50.535547 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ba14bcfe4b406d8051b4397287ca6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.462687 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet f2663906e11549e3aa9ae78d88e645ea with delete_type TABLET_DATA_DELETED (replaced by tablet d4434cb08d0f41fd96283de49c53b3e8) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:50.482914 12560 tablet_service.cc:1511] Processing CreateTablet for tablet a26e6b837f67445c9383599b7e52de7b (DEFAULT_TABLE table=default.Table_87fb2b97ed26421ba052e4bbb187763c [id=765c307ad69a4fb2a6befc0c55508351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.536412 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a26e6b837f67445c9383599b7e52de7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.536614 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886ab0feae1e49629290bc892e518816" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:50.537352 12594 raft_consensus.cc:2393] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
W20260926 19:58:50.543030 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet f2663906e11549e3aa9ae78d88e645ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2663906e11549e3aa9ae78d88e645ea already in progress: creating tablet
I20260926 19:58:50.549998 12858 leader_election.cc:304] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:50.551802 13588 raft_consensus.cc:3060] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:50.554580 13013 raft_consensus.cc:385] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:50.557475 13541 leader_election.cc:290] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:50.557806 12390 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (table default.Table_d6b47ef12b28492bb711ca846f80f6cb [id=c2c079dc87ec480a82aa5b57b110da17]). Expected version 3 got 0
I20260926 19:58:50.558926 12390 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:50.554674 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 0df471a7b6554d4fb5007adb63babe43 (DEFAULT_TABLE table=default.Table_935be674b5a9419296be4ffa3464b082 [id=d2972f37229845e396114c265b371572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.557667 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ef773d1c6694c63a011819024451b02" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:50.559882 12595 raft_consensus.cc:2468] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:50.560851 12695 leader_election.cc:304] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:50.561097 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df471a7b6554d4fb5007adb63babe43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.564206 12568 tablet_service.cc:1511] Processing CreateTablet for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 (DEFAULT_TABLE table=default.Table_1cd57fde79174a2e8092337be2e108c1 [id=f15adbe1da374a8b9b507ff371c1286c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.565112 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.565649 13013 raft_consensus.cc:740] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:50.566291 13013 consensus_queue.cc:260] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.567066 12590 raft_consensus.cc:2468] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:50.559958 12390 catalog_manager.cc:5838] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5 reported cstate change: term changed from 0 to 2, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 2 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } }
I20260926 19:58:50.576802 12695 leader_election.cc:304] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:50.578099 12390 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]). Expected version 2 got 0
I20260926 19:58:50.578433 13542 raft_consensus.cc:2804] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:50.578787 13542 raft_consensus.cc:493] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:50.579104 13542 raft_consensus.cc:3060] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:50.581096 12390 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 3dabe03472b640eca7d88e4c54cc2782 (table default.Table_1dfed69d00a647d1814ee455d0d3b5e2 [id=3f2e4633bd6c4004ab931d817248ed45]). Expected version 1 got 0
I20260926 19:58:50.585878 13587 raft_consensus.cc:2804] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:50.586241 13587 raft_consensus.cc:493] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:50.586553 13587 raft_consensus.cc:3060] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:50.589838 13092 tablet_bootstrap.cc:492] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:50.590310 13092 ts_tablet_manager.cc:1403] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.445s user 0.013s sys 0.000s
I20260926 19:58:50.591939 13542 raft_consensus.cc:515] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.600323 13403 ts_tablet_manager.cc:1929] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:50.600752 13403 log.cc:1199] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/9acbd4e5aa23433d860d399d78485d0d
I20260926 19:58:50.558792 13593 tablet.cc:1751] T 189eaca5fb2b4b0d83a2873e0495e5ff P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:50.609136 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ef773d1c6694c63a011819024451b02" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:50.609925 12593 raft_consensus.cc:3060] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:50.617071 12593 raft_consensus.cc:2468] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:50.617915 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ef773d1c6694c63a011819024451b02" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:50.624945 13542 leader_election.cc:290] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:50.629386 12732 tablet_service.cc:1511] Processing CreateTablet for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 (DEFAULT_TABLE table=default.Table_1cd57fde79174a2e8092337be2e108c1 [id=f15adbe1da374a8b9b507ff371c1286c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.630739 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:50.635178 12696 leader_election.cc:343] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:50.636225 13403 ts_tablet_manager.cc:1950] T 9acbd4e5aa23433d860d399d78485d0d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
W20260926 19:58:50.637269 12382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111) [suppressed 5 similar messages]
I20260926 19:58:50.646498 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 13e75602e3334f599230eb0cfe64e33f (DEFAULT_TABLE table=default.Table_070eaadfa595480ca36eede1de4c5a1f [id=f42573b53e754ef5a05f60f1e8c764ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.647469 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e75602e3334f599230eb0cfe64e33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.648200 12904 tablet_service.cc:1511] Processing CreateTablet for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 (DEFAULT_TABLE table=default.Table_1cd57fde79174a2e8092337be2e108c1 [id=f15adbe1da374a8b9b507ff371c1286c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.593748 13092 raft_consensus.cc:359] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.650131 13092 raft_consensus.cc:385] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:50.650431 13092 raft_consensus.cc:740] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:50.650768 12897 tablet_service.cc:1511] Processing CreateTablet for tablet 13e75602e3334f599230eb0cfe64e33f (DEFAULT_TABLE table=default.Table_070eaadfa595480ca36eede1de4c5a1f [id=f42573b53e754ef5a05f60f1e8c764ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.652734 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e75602e3334f599230eb0cfe64e33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.658373 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.669349 13092 consensus_queue.cc:260] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.671672 13588 raft_consensus.cc:2749] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:50.695653 12381 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 59f7dc92330d400f93f427f3f79ad826 (table default.Table_85695893a5b24687959b83a8360d1443 [id=185449ae1c9744eba4df8bd0f9704abb], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 59f7dc92330d400f93f427f3f79ad826 consensus state has no leader
I20260926 19:58:50.686856 13587 raft_consensus.cc:515] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.710368 13012 tablet_bootstrap.cc:492] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:50.710764 13012 ts_tablet_manager.cc:1403] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.265s user 0.041s sys 0.040s
I20260926 19:58:50.712574 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d11cb23144691809c554c125cad0e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:50.714010 12696 leader_election.cc:343] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:50.716750 13012 raft_consensus.cc:359] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.719283 13092 ts_tablet_manager.cc:1434] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.129s user 0.006s sys 0.000s
I20260926 19:58:50.719977 13012 raft_consensus.cc:385] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:50.720121 13092 tablet_bootstrap.cc:492] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:50.720348 13012 raft_consensus.cc:740] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:50.721113 13012 consensus_queue.cc:260] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.722538 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 174) took 2644 ms. Trace:
I20260926 19:58:50.723114 13012 ts_tablet_manager.cc:1434] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.012s user 0.003s sys 0.002s
I20260926 19:58:50.722889 13403 rpcz_store.cc:276] 0926 19:58:48.078515 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:48.078657 (+ 142us) service_pool.cc:224] Handling call
0926 19:58:49.991337 (+1912680us) ts_tablet_manager.cc:1151] Deleting tablet 9acbd4e5aa23433d860d399d78485d0d
0926 19:58:49.991396 (+ 59us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:50.269322 (+277926us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:50.600208 (+330886us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:50.722481 (+122273us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1912012}
I20260926 19:58:50.724164 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 9acbd4e5aa23433d860d399d78485d0d (table default.Table_e0ad88ad494c42759968cb1dc8501282 [id=6890560cf0104034a9eb9431a38c99ec]) successfully deleted
I20260926 19:58:50.724614 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 178) took 1571 ms. Trace:
I20260926 19:58:50.735762 13012 tablet_bootstrap.cc:492] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:50.738794 12394 catalog_manager.cc:5838] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:50.740797 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]). Expected version 2 got 0
I20260926 19:58:50.624946 13013 ts_tablet_manager.cc:1434] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.261s user 0.005s sys 0.000s
I20260926 19:58:50.731976 12921 raft_consensus.cc:3060] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:50.758033 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (table default.Table_d6b47ef12b28492bb711ca846f80f6cb [id=c2c079dc87ec480a82aa5b57b110da17]). Expected version 3 got 0
I20260926 19:58:50.765898 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:50.765775 13599 consensus_queue.cc:1048] T 1aa82c030ec34ac5a8c448d80cdbae95 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:50.766541 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:50.767277 12394 catalog_manager.cc:5838] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } }
I20260926 19:58:50.768591 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 3dabe03472b640eca7d88e4c54cc2782 (table default.Table_1dfed69d00a647d1814ee455d0d3b5e2 [id=3f2e4633bd6c4004ab931d817248ed45]). Expected version 1 got 0
I20260926 19:58:50.769654 12394 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:50.770678 12394 catalog_manager.cc:5838] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:50.771617 13403 rpcz_store.cc:276] 0926 19:58:49.153332 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:49.153452 (+ 120us) service_pool.cc:224] Handling call
0926 19:58:50.724102 (+1570650us) ts_tablet_manager.cc:1151] Deleting tablet 59f7dc92330d400f93f427f3f79ad826
0926 19:58:50.724155 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:50.724552 (+ 397us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1570123}
I20260926 19:58:50.772249 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 64149f9e4d384ddbb922291ac7de35b7
W20260926 19:58:50.772971 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 59f7dc92330d400f93f427f3f79ad826 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 59f7dc92330d400f93f427f3f79ad826 already in progress: creating tablet
I20260926 19:58:50.773641 13587 leader_election.cc:290] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:50.773738 12396 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:45734 (request call id 252) took 2687 ms. Trace:
I20260926 19:58:50.775830 13012 tablet_bootstrap.cc:654] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:50.774004 12396 rpcz_store.cc:276] 0926 19:58:48.086545 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:48.986294 (+899749us) service_pool.cc:224] Handling call
0926 19:58:49.128004 (+141710us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8876 00007fb36a687b34 00007fb36a687ac2 00007fb36a6879f1 00007fb36a68796c 00007fb36a686c22 00007fb36892d09c 00007fb36892c4d6
0926 19:58:49.142133 (+ 14129us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:49.143678 (+ 1545us) catalog_manager.cc:2854] Modifying in-memory table state
0926 19:58:49.143805 (+ 127us) catalog_manager.cc:2863] Locking tablets
0926 19:58:49.495311 (+351506us) catalog_manager.cc:2877] Removing table and tablets from system table
0926 19:58:49.641775 (+146464us) catalog_manager.cc:2895] Removing table from by-name map
0926 19:58:49.641978 (+ 203us) catalog_manager.cc:2911] Committing in-memory state
0926 19:58:50.772162 (+1130184us) catalog_manager.cc:2915] Aborting table tasks
0926 19:58:50.773671 (+ 1509us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:49.598536 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:49.599085 (+ 549us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:49.599106 (+ 21us) write_op.cc:435] Acquired schema lock
0926 19:58:49.599120 (+ 14us) tablet.cc:662] Decoding operations
0926 19:58:49.599549 (+ 429us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:49.599613 (+ 64us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:49.599648 (+ 35us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:49.600024 (+ 376us) tablet.cc:701] Row locks acquired
0926 19:58:49.600046 (+ 22us) write_op.cc:260] PREPARE: finished
0926 19:58:49.600201 (+ 155us) write_op.cc:270] Start()
0926 19:58:49.600322 (+ 121us) write_op.cc:276] Timestamp: P: 1790452729600142 usec, L: 0
0926 19:58:49.600360 (+ 38us) op_driver.cc:348] REPLICATION: starting
0926 19:58:49.629666 (+ 29306us) log.cc:844] Serialized 774 byte log entry
0926 19:58:49.630276 (+ 610us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36ac5a5e8 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11 00007fb36b0edd3d 00000000005bd4e4 00000000005bd319
0926 19:58:49.639735 (+ 9459us) op_driver.cc:464] REPLICATION: finished
0926 19:58:49.639974 (+ 239us) write_op.cc:301] APPLY: starting
0926 19:58:49.640030 (+ 56us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:49.640139 (+ 109us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:49.640158 (+ 19us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:49.640403 (+ 245us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:49.640541 (+ 138us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:58:49.640587 (+ 46us) write_op.cc:312] APPLY: finished
0926 19:58:49.640752 (+ 165us) log.cc:844] Serialized 29 byte log entry
0926 19:58:49.640902 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.640967 (+ 65us) write_op.cc:454] Released schema lock
0926 19:58:49.640997 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:49.663635 (+ 22638us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.663677 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":322336000,"child_traces":[["op",{"apply.queue_time_us":78,"apply.run_cpu_time_us":1193,"apply.run_wall_time_us":1193,"num_ops":2,"prepare.queue_time_us":101344,"prepare.run_cpu_time_us":3772,"prepare.run_wall_time_us":65063,"raft.queue_time_us":163,"raft.run_cpu_time_us":1034,"raft.run_wall_time_us":2557,"replication_time_us":39246,"spinlock_wait_cycles":65596672}]]}
I20260926 19:58:50.779865 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 64149f9e4d384ddbb922291ac7de35b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:50.780581 13013 tablet_bootstrap.cc:492] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:50.780596 13569 tablet_replica.cc:333] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:50.781392 13569 raft_consensus.cc:2243] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:50.782212 13092 tablet_bootstrap.cc:654] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:50.782275 13569 raft_consensus.cc:2272] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:50.783213 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 64149f9e4d384ddbb922291ac7de35b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:50.784157 13569 ts_tablet_manager.cc:1916] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:50.785027 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 13e75602e3334f599230eb0cfe64e33f (DEFAULT_TABLE table=default.Table_070eaadfa595480ca36eede1de4c5a1f [id=f42573b53e754ef5a05f60f1e8c764ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.786404 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 8f818570b3a24899ae3abe19012efb16 (DEFAULT_TABLE table=default.Table_cc8a261b07da4eb384b6ac40b7c5c5c3 [id=f26485765a7e4fb7980d1337e78a226c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.787631 12390 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_d739366722334ab9a135d008dccea789 [id=02441df5c4e6405fa8a72aa72b16687e] alter complete (version 1)
I20260926 19:58:50.790786 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 8f818570b3a24899ae3abe19012efb16 (DEFAULT_TABLE table=default.Table_cc8a261b07da4eb384b6ac40b7c5c5c3 [id=f26485765a7e4fb7980d1337e78a226c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.791908 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f818570b3a24899ae3abe19012efb16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.794952 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 64149f9e4d384ddbb922291ac7de35b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:50.795873 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 8f818570b3a24899ae3abe19012efb16 (DEFAULT_TABLE table=default.Table_cc8a261b07da4eb384b6ac40b7c5c5c3 [id=f26485765a7e4fb7980d1337e78a226c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.796808 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f818570b3a24899ae3abe19012efb16. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:50.796761 12393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 46) took 3628 ms (client timeout 1000 ms). Trace:
I20260926 19:58:50.798120 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.800061 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f818570b3a24899ae3abe19012efb16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.801839 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.802744 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9411d3bca81641a4b3dfaffb2dc2bb51. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:50.805274 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 22f268b4a1f7423f917731c6b62aab3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22f268b4a1f7423f917731c6b62aab3e already in progress: creating tablet
I20260926 19:58:50.819901 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9411d3bca81641a4b3dfaffb2dc2bb51. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:50.820211 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet f2663906e11549e3aa9ae78d88e645ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2663906e11549e3aa9ae78d88e645ea already in progress: creating tablet
I20260926 19:58:50.821072 13403 tablet_replica.cc:333] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:50.825147 13203 tablet_replica.cc:333] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:50.825889 13203 raft_consensus.cc:2243] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:50.826346 13203 raft_consensus.cc:2272] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:50.828223 13203 ts_tablet_manager.cc:1916] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:50.832620 13403 raft_consensus.cc:2243] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260926 19:58:50.832890 12388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 47) took 2646 ms (client timeout 1000 ms). Trace:
I20260926 19:58:50.833118 13403 raft_consensus.cc:2272] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:50.833202 12388 rpcz_store.cc:269] 0926 19:58:48.186811 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:48.371398 (+184587us) service_pool.cc:224] Handling call
0926 19:58:48.443663 (+ 72265us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:50.832826 (+2389163us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:49.310681 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:49.311102 (+ 421us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:49.311127 (+ 25us) write_op.cc:435] Acquired schema lock
0926 19:58:49.311142 (+ 15us) tablet.cc:662] Decoding operations
0926 19:58:49.311388 (+ 246us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:49.311426 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:49.311451 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:49.311602 (+ 151us) tablet.cc:701] Row locks acquired
0926 19:58:49.312522 (+ 920us) write_op.cc:260] PREPARE: finished
0926 19:58:49.312631 (+ 109us) write_op.cc:270] Start()
0926 19:58:49.312718 (+ 87us) write_op.cc:276] Timestamp: P: 1790452729312587 usec, L: 0
0926 19:58:49.312733 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:49.313302 (+ 569us) log.cc:844] Serialized 536 byte log entry
0926 19:58:49.484965 (+171663us) op_driver.cc:464] REPLICATION: finished
0926 19:58:49.485929 (+ 964us) write_op.cc:301] APPLY: starting
0926 19:58:49.485987 (+ 58us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:49.486071 (+ 84us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:49.486174 (+ 103us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:49.486358 (+ 184us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:49.486495 (+ 137us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:49.486537 (+ 42us) write_op.cc:312] APPLY: finished
0926 19:58:49.486693 (+ 156us) log.cc:844] Serialized 23 byte log entry
0926 19:58:49.486864 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.486925 (+ 61us) write_op.cc:454] Released schema lock
0926 19:58:49.486957 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:49.487175 (+ 218us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.487203 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":15,"spinlock_wait_cycles":159789312,"child_traces":[["op",{"apply.queue_time_us":834,"apply.run_cpu_time_us":1217,"apply.run_wall_time_us":1216,"num_ops":1,"prepare.queue_time_us":333383,"prepare.run_cpu_time_us":1968,"prepare.run_wall_time_us":2845,"replication_time_us":172153}]]}
I20260926 19:58:50.843844 13013 tablet_bootstrap.cc:654] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:50.845803 12395 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_f136506f88074adcb77c8ab2bc88ce51 [id=80c38f5ed2d841bc82af02a3dc13ba60] alter complete (version 2)
I20260926 19:58:50.847857 12353 master_runner.cc:187] Error getting master registration for 127.11.233.253:37913: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111),
I20260926 19:58:50.848275 12353 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:58:50.848907 13092 tablet_bootstrap.cc:492] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:50.849288 13092 ts_tablet_manager.cc:1403] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.129s user 0.009s sys 0.000s
I20260926 19:58:50.852545 13569 ts_tablet_manager.cc:1929] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:50.852939 13569 log.cc:1199] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/64149f9e4d384ddbb922291ac7de35b7
I20260926 19:58:50.858479 13403 ts_tablet_manager.cc:1916] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:50.862026 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d11cb23144691809c554c125cad0e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:50.862512 13569 ts_tablet_manager.cc:1950] T 64149f9e4d384ddbb922291ac7de35b7 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:50.862591 12594 raft_consensus.cc:3060] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:50.863013 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccbbe1a5784e97827a31e9febd8c20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:50.865653 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 734c7713a06449c190105c1478b62251 (DEFAULT_TABLE table=default.Table_e70be55e31fe48c8a83266f2d0937cab [id=e77b8b8297324975b03fcc74e50d3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.867697 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e75602e3334f599230eb0cfe64e33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.869189 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 734c7713a06449c190105c1478b62251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.873083 12594 raft_consensus.cc:2468] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
W20260926 19:58:50.876572 12392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 55) took 2743 ms (client timeout 1000 ms). Trace:
W20260926 19:58:50.877830 12392 rpcz_store.cc:269] 0926 19:58:48.133031 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:48.369363 (+236332us) service_pool.cc:224] Handling call
0926 19:58:48.446042 (+ 76679us) spinlock_profiling.cc:243] Waited 74.4 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:50.876506 (+2430464us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":11,"spinlock_wait_cycles":171036416}
I20260926 19:58:50.878396 12901 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c with delete_type TABLET_DATA_DELETED (replaced by tablet 22f268b4a1f7423f917731c6b62aab3e) from {username='slave'} at 127.0.0.1:33594
W20260926 19:58:50.878654 12389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 54) took 3733 ms (client timeout 1000 ms). Trace:
W20260926 19:58:50.879037 12389 rpcz_store.cc:269] 0926 19:58:47.119671 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.170434 (+ 50763us) service_pool.cc:224] Handling call
0926 19:58:50.853442 (+3683008us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":5}
I20260926 19:58:50.880369 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_24fce316a7434e408e47ec249b8a824f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:50.882025 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 1455b99a65ff44ff933825196708f485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:50.886276 12904 tablet_service.cc:1511] Processing CreateTablet for tablet dbd3a49039354c28b8faf1fd052210ee (DEFAULT_TABLE table=default.Table_e4d93f2b15784ea8b7c7bdd2522017e5 [id=3da4fb37d3cc41e689e4463eb54dbde7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:50.887065 12391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 52) took 1626 ms (client timeout 1000 ms). Trace:
I20260926 19:58:50.887493 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd3a49039354c28b8faf1fd052210ee. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:50.887439 12391 rpcz_store.cc:269] 0926 19:58:49.260435 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:49.530713 (+270278us) service_pool.cc:224] Handling call
0926 19:58:49.540570 (+ 9857us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:50.887005 (+1346435us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23370,"reported_tablets":10,"spinlock_wait_cycles":15190784}
I20260926 19:58:50.888937 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f8331edd1b074ce1b2b43aaf0100626b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:50.896317 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 with delete_type TABLET_DATA_DELETED (replaced by tablet de85e3e17bf340f8895aff3f5421c85c) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:50.897224 13092 raft_consensus.cc:359] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.899399 13092 raft_consensus.cc:385] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:50.900156 13092 raft_consensus.cc:740] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
W20260926 19:58:50.900792 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8331edd1b074ce1b2b43aaf0100626b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:50.896531 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24fce316a7434e408e47ec249b8a824f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:50.900887 13092 consensus_queue.cc:260] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.908231 12387 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_a8870fb0212e43b788b3fced198aed4e [id=2206b9a20656421f8bbd96810835c982] alter complete (version 1)
W20260926 19:58:50.909696 12387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 56) took 1768 ms (client timeout 1000 ms). Trace:
I20260926 19:58:50.913862 13092 ts_tablet_manager.cc:1434] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.064s user 0.005s sys 0.000s
I20260926 19:58:50.914438 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet 886ab0feae1e49629290bc892e518816 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad76fc959e24545bdfad19f988fce93) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:50.897406 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_35146d1cd4df4da594f520e22409c68f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:50.918848 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35146d1cd4df4da594f520e22409c68f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:50.924587 12395 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 48) took 1701 ms (client timeout 1000 ms). Trace:
I20260926 19:58:50.929126 13092 tablet_bootstrap.cc:492] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
W20260926 19:58:50.797066 12393 rpcz_store.cc:269] 0926 19:58:47.168020 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:47.227595 (+ 59575us) service_pool.cc:224] Handling call
0926 19:58:47.297238 (+ 69643us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:50.796666 (+3499428us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:48.254512 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:48.255014 (+ 502us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:48.255045 (+ 31us) write_op.cc:435] Acquired schema lock
0926 19:58:48.255063 (+ 18us) tablet.cc:662] Decoding operations
0926 19:58:48.255304 (+ 241us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:48.255350 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:48.255383 (+ 33us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:48.255563 (+ 180us) tablet.cc:701] Row locks acquired
0926 19:58:48.255582 (+ 19us) write_op.cc:260] PREPARE: finished
0926 19:58:48.255703 (+ 121us) write_op.cc:270] Start()
0926 19:58:48.255803 (+ 100us) write_op.cc:276] Timestamp: P: 1790452728255650 usec, L: 0
0926 19:58:48.255824 (+ 21us) op_driver.cc:348] REPLICATION: starting
0926 19:58:48.342562 (+ 86738us) log.cc:844] Serialized 536 byte log entry
0926 19:58:48.350305 (+ 7743us) op_driver.cc:464] REPLICATION: finished
0926 19:58:48.350729 (+ 424us) write_op.cc:301] APPLY: starting
0926 19:58:48.350799 (+ 70us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:48.350922 (+ 123us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:48.351061 (+ 139us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:48.351293 (+ 232us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:48.351458 (+ 165us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:48.351517 (+ 59us) write_op.cc:312] APPLY: finished
0926 19:58:48.351732 (+ 215us) log.cc:844] Serialized 23 byte log entry
0926 19:58:48.351886 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.351954 (+ 68us) write_op.cc:454] Released schema lock
0926 19:58:48.351985 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:48.462402 (+110417us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:48.462436 (+ 34us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 19:58:49.197166 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:49.279687 (+ 82521us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:49.279718 (+ 31us) write_op.cc:435] Acquired schema lock
0926 19:58:49.279732 (+ 14us) tablet.cc:662] Decoding operations
0926 19:58:49.280028 (+ 296us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:49.280071 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:49.280096 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:49.280240 (+ 144us) tablet.cc:701] Row locks acquired
0926 19:58:49.280255 (+ 15us) write_op.cc:260] PREPARE: finished
0926 19:58:49.280375 (+ 120us) write_op.cc:270] Start()
0926 19:58:49.280457 (+ 82us) write_op.cc:276] Timestamp: P: 1790452729280328 usec, L: 0
0926 19:58:49.280473 (+ 16us) op_driver.cc:348] REPLICATION: starting
0926 19:58:49.280993 (+ 520us) log.cc:844] Serialized 356 byte log entry
0926 19:58:49.483564 (+202571us) op_driver.cc:464] REPLICATION: finished
0926 19:58:49.483866 (+ 302us) write_op.cc:301] APPLY: starting
0926 19:58:49.483934 (+ 68us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:49.484042 (+ 108us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:49.484130 (+ 88us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:49.484302 (+ 172us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:49.484432 (+ 130us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:49.484477 (+ 45us) write_op.cc:312] APPLY: finished
0926 19:58:49.484654 (+ 177us) log.cc:844] Serialized 23 byte log entry
0926 19:58:49.484796 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.484869 (+ 73us) write_op.cc:454] Released schema lock
0926 19:58:49.484899 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:49.485120 (+ 221us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:49.485146 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":149670784,"child_traces":[["op",{"apply.queue_time_us":256,"apply.run_cpu_time_us":1449,"apply.run_wall_time_us":1449,"num_ops":1,"prepare.queue_time_us":789910,"prepare.run_cpu_time_us":2396,"prepare.run_wall_time_us":207855,"raft.queue_time_us":334,"raft.run_cpu_time_us":732,"raft.run_wall_time_us":733,"replication_time_us":94364}],["op",{"apply.queue_time_us":110,"apply.run_cpu_time_us":1225,"apply.run_wall_time_us":1221,"mutex_wait_us":533,"num_ops":1,"prepare.queue_time_us":833078,"prepare.run_cpu_time_us":2103,"prepare.run_wall_time_us":113406,"raft.queue_time_us":28452,"raft.run_cpu_time_us":731,"raft.run_wall_time_us":732,"replication_time_us":202945}]]}
I20260926 19:58:50.930815 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_09d6cf0518f640eaa9d4cf503b55422b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:50.887696 12565 tablet_service.cc:1558] Processing DeleteTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d0583424ef64d9ea3beeb1c01ceafba) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:50.887930 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet 886ab0feae1e49629290bc892e518816 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad76fc959e24545bdfad19f988fce93) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:50.888819 12564 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e337767be84507acdecfb189705bdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:50.889140 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 734c7713a06449c190105c1478b62251 (DEFAULT_TABLE table=default.Table_e70be55e31fe48c8a83266f2d0937cab [id=e77b8b8297324975b03fcc74e50d3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.917701 12558 tablet_service.cc:1511] Processing CreateTablet for tablet dbd3a49039354c28b8faf1fd052210ee (DEFAULT_TABLE table=default.Table_e4d93f2b15784ea8b7c7bdd2522017e5 [id=3da4fb37d3cc41e689e4463eb54dbde7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:50.937307 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a8870fb0212e43b788b3fced198aed4e" } modify_external_catalogs: true
I20260926 19:58:50.938081 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd3a49039354c28b8faf1fd052210ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.949898 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 734c7713a06449c190105c1478b62251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:50.968482 13092 tablet_bootstrap.cc:654] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:50.977998 13588 raft_consensus.cc:493] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:50.978440 13588 raft_consensus.cc:515] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:50.981146 13588 leader_election.cc:290] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:50.982122 13403 ts_tablet_manager.cc:1929] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:50.982463 13403 log.cc:1199] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/64149f9e4d384ddbb922291ac7de35b7
I20260926 19:58:50.984064 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e337767be84507acdecfb189705bdd" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:50.988036 13013 tablet_bootstrap.cc:492] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:50.988471 13013 ts_tablet_manager.cc:1403] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.208s user 0.011s sys 0.002s
I20260926 19:58:50.993952 13403 ts_tablet_manager.cc:1950] T 64149f9e4d384ddbb922291ac7de35b7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:50.836393 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_d739366722334ab9a135d008dccea789" } new_table_name: "default.Table_979fb16236da4673983d947c83506fd3" modify_external_catalogs: true
W20260926 19:58:50.933485 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09d6cf0518f640eaa9d4cf503b55422b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:50.996308 13403 tablet_replica.cc:333] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:50.996520 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 64149f9e4d384ddbb922291ac7de35b7 (table default.Table_c489e926efca4dafb56b6642f8404c2b [id=00773dee014a47fca49e7dfd9ed0d584]) successfully deleted
I20260926 19:58:50.997026 13403 raft_consensus.cc:2243] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:50.997442 13403 raft_consensus.cc:2272] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:51.000674 13403 ts_tablet_manager.cc:1916] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:50.910027 12387 rpcz_store.cc:269] 0926 19:58:49.141472 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:49.157214 (+ 15742us) service_pool.cc:224] Handling call
0926 19:58:50.909633 (+1752419us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:50.056836 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:50.057202 (+ 366us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:50.057226 (+ 24us) write_op.cc:435] Acquired schema lock
0926 19:58:50.057239 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:50.057484 (+ 245us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:50.057568 (+ 84us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:50.057589 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:50.057741 (+ 152us) tablet.cc:701] Row locks acquired
0926 19:58:50.057753 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:50.057847 (+ 94us) write_op.cc:270] Start()
0926 19:58:50.057929 (+ 82us) write_op.cc:276] Timestamp: P: 1790452730057807 usec, L: 0
0926 19:58:50.057944 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:50.058391 (+ 447us) log.cc:844] Serialized 536 byte log entry
0926 19:58:50.067041 (+ 8650us) op_driver.cc:464] REPLICATION: finished
0926 19:58:50.067420 (+ 379us) write_op.cc:301] APPLY: starting
0926 19:58:50.067472 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:50.067561 (+ 89us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:50.067659 (+ 98us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:50.067796 (+ 137us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:50.067917 (+ 121us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:50.067960 (+ 43us) write_op.cc:312] APPLY: finished
0926 19:58:50.068115 (+ 155us) log.cc:844] Serialized 23 byte log entry
0926 19:58:50.068241 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.068298 (+ 57us) write_op.cc:454] Released schema lock
0926 19:58:50.068326 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:50.068533 (+ 207us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.068559 (+ 26us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 19:58:50.891375 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:50.891918 (+ 543us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:50.891955 (+ 37us) write_op.cc:435] Acquired schema lock
0926 19:58:50.891975 (+ 20us) tablet.cc:662] Decoding operations
0926 19:58:50.892322 (+ 347us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:50.892381 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:50.892415 (+ 34us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:50.892600 (+ 185us) tablet.cc:701] Row locks acquired
0926 19:58:50.892620 (+ 20us) write_op.cc:260] PREPARE: finished
0926 19:58:50.892769 (+ 149us) write_op.cc:270] Start()
0926 19:58:50.892860 (+ 91us) write_op.cc:276] Timestamp: P: 1790452730892703 usec, L: 0
0926 19:58:50.892873 (+ 13us) op_driver.cc:348] REPLICATION: starting
0926 19:58:50.893325 (+ 452us) log.cc:844] Serialized 356 byte log entry
0926 19:58:50.902406 (+ 9081us) op_driver.cc:464] REPLICATION: finished
0926 19:58:50.902722 (+ 316us) write_op.cc:301] APPLY: starting
0926 19:58:50.902777 (+ 55us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:50.902875 (+ 98us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:50.902974 (+ 99us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:50.903109 (+ 135us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:50.903236 (+ 127us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:50.903275 (+ 39us) write_op.cc:312] APPLY: finished
0926 19:58:50.903432 (+ 157us) log.cc:844] Serialized 23 byte log entry
0926 19:58:50.903561 (+ 129us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.903621 (+ 60us) write_op.cc:454] Released schema lock
0926 19:58:50.903649 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:50.903855 (+ 206us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.903877 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":246,"apply.run_cpu_time_us":1087,"apply.run_wall_time_us":1087,"num_ops":1,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":1839,"prepare.run_wall_time_us":1839,"raft.queue_time_us":2634,"raft.run_cpu_time_us":658,"raft.run_wall_time_us":658,"replication_time_us":9009}],["op",{"apply.queue_time_us":170,"apply.run_cpu_time_us":1090,"apply.run_wall_time_us":1090,"num_ops":1,"prepare.queue_time_us":112,"prepare.run_cpu_time_us":2290,"prepare.run_wall_time_us":3264,"raft.queue_time_us":1182,"raft.run_cpu_time_us":630,"raft.run_wall_time_us":626,"replication_time_us":9443}]]}
I20260926 19:58:51.009522 13013 raft_consensus.cc:359] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:50.924978 12395 rpcz_store.cc:269] 0926 19:58:49.223128 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:49.519321 (+296193us) service_pool.cc:224] Handling call
0926 19:58:50.924516 (+1405195us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:50.346366 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:50.346778 (+ 412us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:50.346800 (+ 22us) write_op.cc:435] Acquired schema lock
0926 19:58:50.346811 (+ 11us) tablet.cc:662] Decoding operations
0926 19:58:50.347058 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:50.347098 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:50.347120 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:50.347263 (+ 143us) tablet.cc:701] Row locks acquired
0926 19:58:50.347279 (+ 16us) write_op.cc:260] PREPARE: finished
0926 19:58:50.347381 (+ 102us) write_op.cc:270] Start()
0926 19:58:50.347459 (+ 78us) write_op.cc:276] Timestamp: P: 1790452730347326 usec, L: 0
0926 19:58:50.347473 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:50.357208 (+ 9735us) log.cc:844] Serialized 536 byte log entry
0926 19:58:50.388459 (+ 31251us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b58000809d0. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36abb896c 00007fb36abbf283 00007fb36abbf209 00007fb36abbf199 00007fb36abbf161 00007fb36abbe32d 00000000005bd4e4 00000000005bd319 00007fb363afc3e5 00007fb363afe751
0926 19:58:50.388719 (+ 260us) op_driver.cc:464] REPLICATION: finished
0926 19:58:50.389063 (+ 344us) write_op.cc:301] APPLY: starting
0926 19:58:50.389115 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:50.389203 (+ 88us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:50.389222 (+ 19us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:50.389363 (+ 141us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:50.389489 (+ 126us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:50.389554 (+ 65us) write_op.cc:312] APPLY: finished
0926 19:58:50.389713 (+ 159us) log.cc:844] Serialized 23 byte log entry
0926 19:58:50.389861 (+ 148us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.389920 (+ 59us) write_op.cc:454] Released schema lock
0926 19:58:50.389951 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:50.390168 (+ 217us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.390192 (+ 24us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 19:58:50.472387 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:50.472754 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:50.472789 (+ 35us) write_op.cc:435] Acquired schema lock
0926 19:58:50.472801 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:50.473035 (+ 234us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:50.473074 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:50.473095 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:50.473228 (+ 133us) tablet.cc:701] Row locks acquired
0926 19:58:50.473240 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:50.473344 (+ 104us) write_op.cc:270] Start()
0926 19:58:50.473423 (+ 79us) write_op.cc:276] Timestamp: P: 1790452730473282 usec, L: 0
0926 19:58:50.473438 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:50.473923 (+ 485us) log.cc:844] Serialized 356 byte log entry
0926 19:58:50.483815 (+ 9892us) op_driver.cc:464] REPLICATION: finished
0926 19:58:50.484662 (+ 847us) write_op.cc:301] APPLY: starting
0926 19:58:50.484712 (+ 50us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:50.484811 (+ 99us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:50.484899 (+ 88us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:50.485039 (+ 140us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:50.485164 (+ 125us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:50.485204 (+ 40us) write_op.cc:312] APPLY: finished
0926 19:58:50.485841 (+ 637us) log.cc:844] Serialized 23 byte log entry
0926 19:58:50.486028 (+ 187us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.486113 (+ 85us) write_op.cc:454] Released schema lock
0926 19:58:50.486160 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:50.506537 (+ 20377us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.506562 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":202,"apply.run_cpu_time_us":1074,"apply.run_wall_time_us":1074,"mutex_wait_us":155687,"num_ops":1,"prepare.queue_time_us":175923,"prepare.run_cpu_time_us":1964,"prepare.run_wall_time_us":11167,"raft.queue_time_us":2810,"raft.run_cpu_time_us":3560,"raft.run_wall_time_us":32548,"replication_time_us":41169,"spinlock_wait_cycles":63895296}],["op",{"apply.queue_time_us":707,"apply.run_cpu_time_us":1723,"apply.run_wall_time_us":1722,"num_ops":1,"prepare.queue_time_us":77154,"prepare.run_cpu_time_us":1830,"prepare.run_wall_time_us":34129,"raft.queue_time_us":79,"raft.run_cpu_time_us":692,"raft.run_wall_time_us":22160,"replication_time_us":10275}]]}
I20260926 19:58:51.027422 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_87fb2b97ed26421ba052e4bbb187763c" } new_table_name: "default.Table_5284d05a66f641f1858d0b6dbf51ae4b" modify_external_catalogs: true
I20260926 19:58:51.031778 13403 ts_tablet_manager.cc:1929] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:51.032142 13403 log.cc:1199] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/27ccbbe1a5784e97827a31e9febd8c20
I20260926 19:58:51.032915 13403 ts_tablet_manager.cc:1950] T 27ccbbe1a5784e97827a31e9febd8c20 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:51.045737 13092 tablet_bootstrap.cc:492] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:51.046200 13092 ts_tablet_manager.cc:1403] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.117s user 0.020s sys 0.008s
I20260926 19:58:51.049244 13092 raft_consensus.cc:359] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.049949 13092 raft_consensus.cc:385] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:51.050246 13092 raft_consensus.cc:740] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:51.051577 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 64149f9e4d384ddbb922291ac7de35b7 (table default.Table_c489e926efca4dafb56b6642f8404c2b [id=00773dee014a47fca49e7dfd9ed0d584]) successfully deleted
I20260926 19:58:51.051368 13092 consensus_queue.cc:260] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.055332 12695 leader_election.cc:304] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:51.057710 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet 886ab0feae1e49629290bc892e518816 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad76fc959e24545bdfad19f988fce93) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.060060 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 734c7713a06449c190105c1478b62251 (DEFAULT_TABLE table=default.Table_e70be55e31fe48c8a83266f2d0937cab [id=e77b8b8297324975b03fcc74e50d3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.066108 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 734c7713a06449c190105c1478b62251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.069597 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 27ccbbe1a5784e97827a31e9febd8c20 (table default.Table_8105d15ef7194aeb8eec0f7e42fa5ef1 [id=36e060a7ae90436d8c3917819816609c]) successfully deleted
W20260926 19:58:51.070690 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 0fd184e6295d47aeba809be6aab9ce1c because the tablet was not found. No further retry: Not found: Tablet not found: 0fd184e6295d47aeba809be6aab9ce1c
I20260926 19:58:51.075297 12921 raft_consensus.cc:3060] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:51.084358 12734 tablet_service.cc:1511] Processing CreateTablet for tablet dbd3a49039354c28b8faf1fd052210ee (DEFAULT_TABLE table=default.Table_e4d93f2b15784ea8b7c7bdd2522017e5 [id=3da4fb37d3cc41e689e4463eb54dbde7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.084761 13403 tablet_replica.cc:333] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:51.085682 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd3a49039354c28b8faf1fd052210ee. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:51.094358 12380 catalog_manager.cc:5136] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): alter failed for tablet 64149f9e4d384ddbb922291ac7de35b7 (table default.Table_c489e926efca4dafb56b6642f8404c2b [id=00773dee014a47fca49e7dfd9ed0d584]),no further retry: Not found: Tablet not found: 64149f9e4d384ddbb922291ac7de35b7
I20260926 19:58:51.096151 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.097343 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9411d3bca81641a4b3dfaffb2dc2bb51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.099207 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 189eaca5fb2b4b0d83a2873e0495e5ff (table default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591]). Expected version 2 got 0
I20260926 19:58:51.101061 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (table default.Table_d6b47ef12b28492bb711ca846f80f6cb [id=c2c079dc87ec480a82aa5b57b110da17]). Expected version 3 got 0
I20260926 19:58:51.105909 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a8870fb0212e43b788b3fced198aed4e" } modify_external_catalogs: true
I20260926 19:58:51.106503 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c with delete_type TABLET_DATA_DELETED (replaced by tablet 22f268b4a1f7423f917731c6b62aab3e) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.107544 13092 ts_tablet_manager.cc:1434] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.061s user 0.007s sys 0.000s
I20260926 19:58:51.107803 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e337767be84507acdecfb189705bdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.108637 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet cd92a1a62ab24647bef7c9eefdf9c9d8 with delete_type TABLET_DATA_DELETED (replaced by tablet de85e3e17bf340f8895aff3f5421c85c) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.111479 13092 tablet_bootstrap.cc:492] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:51.111925 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet 1455b99a65ff44ff933825196708f485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.116726 13092 tablet_bootstrap.cc:654] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:51.129665 13403 raft_consensus.cc:2243] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:51.130776 13403 raft_consensus.cc:2272] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:51.134871 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 80011.4us
W20260926 19:58:51.141029 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 57075.8us
I20260926 19:58:51.145648 13013 raft_consensus.cc:385] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:51.145939 13013 raft_consensus.cc:740] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:51.146692 13013 consensus_queue.cc:260] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.154664 13593 tablet.cc:1751] T 3dabe03472b640eca7d88e4c54cc2782 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:51.159243 13590 tablet.cc:1751] T 3dabe03472b640eca7d88e4c54cc2782 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:51.163973 13013 ts_tablet_manager.cc:1434] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.175s user 0.005s sys 0.000s
I20260926 19:58:51.164867 13013 tablet_bootstrap.cc:492] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:51.184715 12393 master_service.cc:960] ReplaceTablet: received request to replace tablet 3066a65ce44b4387a6f96124d9f476fb from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:51.185659 12389 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45734 (request call id 263) took 1449 ms. Trace:
I20260926 19:58:51.186079 12389 rpcz_store.cc:276] 0926 19:58:49.736341 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.879927 (+1143586us) service_pool.cc:224] Handling call
0926 19:58:51.104456 (+224529us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:58:51.105999 (+ 1543us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:58:51.123529 (+ 17530us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:58:51.185335 (+ 61806us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:58:51.185604 (+ 269us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:51.107441 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:51.107858 (+ 417us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:51.107884 (+ 26us) write_op.cc:435] Acquired schema lock
0926 19:58:51.107898 (+ 14us) tablet.cc:662] Decoding operations
0926 19:58:51.108189 (+ 291us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:51.108234 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:51.108258 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:51.108472 (+ 214us) tablet.cc:701] Row locks acquired
0926 19:58:51.108485 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:51.108603 (+ 118us) write_op.cc:270] Start()
0926 19:58:51.108694 (+ 91us) write_op.cc:276] Timestamp: P: 1790452731108557 usec, L: 0
0926 19:58:51.108725 (+ 31us) op_driver.cc:348] REPLICATION: starting
0926 19:58:51.109185 (+ 460us) log.cc:844] Serialized 433 byte log entry
0926 19:58:51.120611 (+ 11426us) op_driver.cc:464] REPLICATION: finished
0926 19:58:51.120866 (+ 255us) write_op.cc:301] APPLY: starting
0926 19:58:51.120934 (+ 68us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:51.121063 (+ 129us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:51.121162 (+ 99us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:51.121821 (+ 659us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:51.121988 (+ 167us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:58:51.122044 (+ 56us) write_op.cc:312] APPLY: finished
0926 19:58:51.122240 (+ 196us) log.cc:844] Serialized 29 byte log entry
0926 19:58:51.122395 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.122466 (+ 71us) write_op.cc:454] Released schema lock
0926 19:58:51.122507 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:51.122750 (+ 243us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.122788 (+ 38us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":1835,"apply.run_wall_time_us":1840,"num_ops":2,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":2037,"prepare.run_wall_time_us":2038,"raft.queue_time_us":227,"raft.run_cpu_time_us":889,"raft.run_wall_time_us":889,"replication_time_us":11802}]]}
I20260926 19:58:51.191438 13569 tablet_replica.cc:333] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:51.193310 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56e337767be84507acdecfb189705bdd" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:51.195698 12857 leader_election.cc:304] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:51.196676 13588 raft_consensus.cc:2749] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:51.189857 12391 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45734 (request call id 264) took 1408 ms. Trace:
I20260926 19:58:51.199487 12391 rpcz_store.cc:276] 0926 19:58:49.781889 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.888565 (+1106676us) service_pool.cc:224] Handling call
0926 19:58:50.977465 (+ 88900us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:58:51.011442 (+ 33977us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:58:51.024851 (+ 13409us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:58:51.189494 (+164643us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:58:51.189800 (+ 306us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:51.012693 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:51.013187 (+ 494us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:51.013224 (+ 37us) write_op.cc:435] Acquired schema lock
0926 19:58:51.013244 (+ 20us) tablet.cc:662] Decoding operations
0926 19:58:51.013591 (+ 347us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:51.013626 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:51.013650 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:51.013923 (+ 273us) tablet.cc:701] Row locks acquired
0926 19:58:51.013944 (+ 21us) write_op.cc:260] PREPARE: finished
0926 19:58:51.014088 (+ 144us) write_op.cc:270] Start()
0926 19:58:51.014193 (+ 105us) write_op.cc:276] Timestamp: P: 1790452731014013 usec, L: 0
0926 19:58:51.014215 (+ 22us) op_driver.cc:348] REPLICATION: starting
0926 19:58:51.014838 (+ 623us) log.cc:844] Serialized 433 byte log entry
0926 19:58:51.022017 (+ 7179us) op_driver.cc:464] REPLICATION: finished
0926 19:58:51.022276 (+ 259us) write_op.cc:301] APPLY: starting
0926 19:58:51.022334 (+ 58us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:51.022461 (+ 127us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:51.022571 (+ 110us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:51.022982 (+ 411us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:51.023128 (+ 146us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:58:51.023174 (+ 46us) write_op.cc:312] APPLY: finished
0926 19:58:51.023353 (+ 179us) log.cc:844] Serialized 29 byte log entry
0926 19:58:51.023510 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.023578 (+ 68us) write_op.cc:454] Released schema lock
0926 19:58:51.023610 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:51.023865 (+ 255us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.023906 (+ 41us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":1549,"apply.run_wall_time_us":1549,"num_ops":2,"prepare.queue_time_us":99,"prepare.run_cpu_time_us":2471,"prepare.run_wall_time_us":2471,"raft.queue_time_us":224,"raft.run_cpu_time_us":886,"raft.run_wall_time_us":886,"replication_time_us":7701}]]}
W20260926 19:58:51.190992 12395 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a26e6b837f67445c9383599b7e52de7b consensus state has no leader
I20260926 19:58:51.205622 13587 raft_consensus.cc:2804] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:51.206266 13587 raft_consensus.cc:697] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:51.209589 13585 tablet.cc:1751] T 0a7d5e580f5645ab95fa03d5f669dc32 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:51.212443 12924 raft_consensus.cc:3060] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:51.218339 12389 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_0e1b1c5a52dc4176b9b867aa57c3ff1c" } modify_external_catalogs: true
I20260926 19:58:51.220216 12389 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_0e1b1c5a52dc4176b9b867aa57c3ff1c" } modify_external_catalogs: true
I20260926 19:58:51.224457 12392 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45734 (request call id 265) took 1247 ms. Trace:
I20260926 19:58:51.224797 12392 rpcz_store.cc:276] 0926 19:58:49.977596 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.896986 (+919390us) service_pool.cc:224] Handling call
0926 19:58:51.177894 (+280908us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:58:51.179264 (+ 1370us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:58:51.223825 (+ 44561us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:58:51.224127 (+ 302us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:58:51.224400 (+ 273us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:51.182584 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:51.182946 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:51.182967 (+ 21us) write_op.cc:435] Acquired schema lock
0926 19:58:51.182979 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:51.183209 (+ 230us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:51.183241 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:51.183265 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:51.183451 (+ 186us) tablet.cc:701] Row locks acquired
0926 19:58:51.183463 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:51.183584 (+ 121us) write_op.cc:270] Start()
0926 19:58:51.183669 (+ 85us) write_op.cc:276] Timestamp: P: 1790452731183514 usec, L: 0
0926 19:58:51.183684 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:51.214754 (+ 31070us) log.cc:844] Serialized 433 byte log entry
0926 19:58:51.215374 (+ 620us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36ac5a5e8 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11 00007fb36b0edd3d 00000000005bd4e4 00000000005bd319
0926 19:58:51.221891 (+ 6517us) op_driver.cc:464] REPLICATION: finished
0926 19:58:51.222245 (+ 354us) write_op.cc:301] APPLY: starting
0926 19:58:51.222301 (+ 56us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:51.222414 (+ 113us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:51.222432 (+ 18us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:51.222929 (+ 497us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:51.223102 (+ 173us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:58:51.223147 (+ 45us) write_op.cc:312] APPLY: finished
0926 19:58:51.223345 (+ 198us) log.cc:844] Serialized 29 byte log entry
0926 19:58:51.223496 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.223565 (+ 69us) write_op.cc:454] Released schema lock
0926 19:58:51.223604 (+ 39us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":175,"num_ops":2,"prepare.queue_time_us":85,"prepare.run_cpu_time_us":3028,"prepare.run_wall_time_us":33040,"raft.queue_time_us":599,"raft.run_cpu_time_us":804,"raft.run_wall_time_us":805,"replication_time_us":38113,"spinlock_wait_cycles":70258816}]]}
I20260926 19:58:51.224220 13590 tablet.cc:1751] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:51.237641 13587 consensus_queue.cc:237] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.245702 13569 raft_consensus.cc:2243] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:51.246393 13569 raft_consensus.cc:2272] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:51.248999 13569 ts_tablet_manager.cc:1916] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:51.249475 13012 tablet_bootstrap.cc:492] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:51.250028 13012 ts_tablet_manager.cc:1403] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.515s user 0.026s sys 0.033s
I20260926 19:58:51.253490 13012 raft_consensus.cc:359] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.254241 13012 raft_consensus.cc:385] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:51.254508 13012 raft_consensus.cc:740] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:51.255050 13012 consensus_queue.cc:260] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.257236 13012 ts_tablet_manager.cc:1434] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260926 19:58:51.268020 13613 consensus_queue.cc:1048] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:51.272306 13585 tablet.cc:1751] T 3dabe03472b640eca7d88e4c54cc2782 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:51.278687 13613 consensus_queue.cc:1048] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:51.279309 13569 ts_tablet_manager.cc:1929] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:51.279978 12695 leader_election.cc:304] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:51.282363 13612 raft_consensus.cc:2804] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:51.279812 13569 log.cc:1199] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/886ab0feae1e49629290bc892e518816
I20260926 19:58:51.282799 13612 raft_consensus.cc:697] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:51.283717 13569 ts_tablet_manager.cc:1950] T 886ab0feae1e49629290bc892e518816 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:51.283645 13612 consensus_queue.cc:237] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.299368 13403 ts_tablet_manager.cc:1916] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:51.321414 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:51.322631 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:51.323248 12396 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 3dabe03472b640eca7d88e4c54cc2782 (table default.Table_1dfed69d00a647d1814ee455d0d3b5e2 [id=3f2e4633bd6c4004ab931d817248ed45]). Expected version 1 got 0
I20260926 19:58:51.324357 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 886ab0feae1e49629290bc892e518816 (table default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189]) successfully deleted
I20260926 19:58:51.325127 13569 tablet_replica.cc:333] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:51.318475 13012 tablet_bootstrap.cc:492] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:51.116868 13203 ts_tablet_manager.cc:1929] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:51.333741 13203 log.cc:1199] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/64149f9e4d384ddbb922291ac7de35b7
I20260926 19:58:51.334784 13203 ts_tablet_manager.cc:1950] T 64149f9e4d384ddbb922291ac7de35b7 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:51.354420 12392 master_service.cc:960] ReplaceTablet: received request to replace tablet 56e337767be84507acdecfb189705bdd from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:51.390655 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_0f5f1b1ec9ab499a9419665ce344105e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:51.392167 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f5f1b1ec9ab499a9419665ce344105e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:51.394091 13569 raft_consensus.cc:2243] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:51.394809 13569 raft_consensus.cc:2272] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:51.396909 13569 ts_tablet_manager.cc:1916] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:44.330097 13353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:51.399583 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 64149f9e4d384ddbb922291ac7de35b7 (table default.Table_c489e926efca4dafb56b6642f8404c2b [id=00773dee014a47fca49e7dfd9ed0d584]) successfully deleted
W20260926 19:58:45.644923 13342 debug-util.cc:398] Leaking SignalData structure 0x7b0800034de0 after lost signal to thread 13130
W20260926 19:58:51.390606 13130 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 7.070s user 0.712s sys 1.280s
W20260926 19:58:51.403710 13130 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 7.083s user 0.712s sys 1.280s
I20260926 19:58:51.412979 13012 tablet_bootstrap.cc:654] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:51.412684 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b70cb5834613437fab92972194f769da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:51.414173 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b70cb5834613437fab92972194f769da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:51.419749 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_8f5fde2cc891426eb3123a67b46f1951" } new_table_name: "default.Table_b294fbdc488a45a3b54ff6e45272ef72" modify_external_catalogs: true
I20260926 19:58:51.420787 13203 tablet_replica.cc:333] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:51.421805 13203 raft_consensus.cc:2243] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:51.422856 13203 pending_rounds.cc:70] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:51.423182 13203 pending_rounds.cc:77] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694374627479552 op_type: NO_OP noop_request { }
I20260926 19:58:51.423676 13203 raft_consensus.cc:2889] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:51.436084 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_063a6e0131614ece86586e6991c8efe8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:51.439455 13590 tablet.cc:1751] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:51.441960 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 381320b4d2844e2289a5ae390e6b0503
W20260926 19:58:51.444703 13623 log.cc:927] Time spent T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9: Append to log took a long time: real 0.051s user 0.000s sys 0.001s
W20260926 19:58:51.444947 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_063a6e0131614ece86586e6991c8efe8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:51.454324 12390 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:45734 (request call id 267) took 1039 ms. Trace:
I20260926 19:58:51.456053 13542 raft_consensus.cc:493] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:51.456204 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet 381320b4d2844e2289a5ae390e6b0503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.456508 13542 raft_consensus.cc:515] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.458511 13542 leader_election.cc:290] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:51.466773 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:51.472911 12696 leader_election.cc:343] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:51.485630 13130 server_base.cc:1061] running on GCE node
W20260926 19:58:51.487222 13632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:58:51.488651 13130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:58:51.492976 13092 tablet_bootstrap.cc:492] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:51.493454 13092 ts_tablet_manager.cc:1403] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.382s user 0.007s sys 0.010s
I20260926 19:58:51.457239 12390 rpcz_store.cc:276] 0926 19:58:50.415164 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.936979 (+521815us) service_pool.cc:224] Handling call
0926 19:58:51.106402 (+169423us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:51.107913 (+ 1511us) catalog_manager.cc:2854] Modifying in-memory table state
0926 19:58:51.108028 (+ 115us) catalog_manager.cc:2863] Locking tablets
0926 19:58:51.406585 (+298557us) catalog_manager.cc:2877] Removing table and tablets from system table
0926 19:58:51.441572 (+ 34987us) catalog_manager.cc:2895] Removing table from by-name map
0926 19:58:51.441752 (+ 180us) catalog_manager.cc:2911] Committing in-memory state
0926 19:58:51.441864 (+ 112us) catalog_manager.cc:2915] Aborting table tasks
0926 19:58:51.454268 (+ 12404us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:51.425662 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:51.426018 (+ 356us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:51.426041 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:58:51.426053 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:51.426341 (+ 288us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:51.426378 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:51.426400 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:51.426645 (+ 245us) tablet.cc:701] Row locks acquired
0926 19:58:51.426658 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:51.426894 (+ 236us) write_op.cc:270] Start()
0926 19:58:51.426974 (+ 80us) write_op.cc:276] Timestamp: P: 1790452731426830 usec, L: 0
0926 19:58:51.426990 (+ 16us) op_driver.cc:348] REPLICATION: starting
0926 19:58:51.427458 (+ 468us) log.cc:844] Serialized 774 byte log entry
0926 19:58:51.434274 (+ 6816us) op_driver.cc:464] REPLICATION: finished
0926 19:58:51.434697 (+ 423us) write_op.cc:301] APPLY: starting
0926 19:58:51.434749 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:51.434857 (+ 108us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:51.434950 (+ 93us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:51.435163 (+ 213us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:51.435286 (+ 123us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:58:51.435329 (+ 43us) write_op.cc:312] APPLY: finished
0926 19:58:51.435503 (+ 174us) log.cc:844] Serialized 29 byte log entry
0926 19:58:51.435634 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.435695 (+ 61us) write_op.cc:454] Released schema lock
0926 19:58:51.435724 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:51.435938 (+ 214us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.435961 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":288,"apply.run_cpu_time_us":1211,"apply.run_wall_time_us":1210,"num_ops":2,"prepare.queue_time_us":17429,"prepare.run_cpu_time_us":2110,"prepare.run_wall_time_us":3078,"raft.queue_time_us":147,"raft.run_cpu_time_us":714,"raft.run_wall_time_us":714,"replication_time_us":7200}]]}
I20260926 19:58:51.494829 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 381320b4d2844e2289a5ae390e6b0503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:51.495995 13092 raft_consensus.cc:359] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.496651 13092 raft_consensus.cc:385] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:51.469754 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 0a7d5e580f5645ab95fa03d5f669dc32 (table default.Table_d6b47ef12b28492bb711ca846f80f6cb [id=c2c079dc87ec480a82aa5b57b110da17]). Expected version 3 got 0
I20260926 19:58:51.496996 13092 raft_consensus.cc:740] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:51.497411 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
W20260926 19:58:51.481988 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3df392d277a447aabf4320a7d9a43880 has no consensus state
I20260926 19:58:51.459515 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_8b39d71ccfc048848d34cf14d1f2e1fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:51.499487 12388 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:45734 (request call id 262) took 1770 ms. Trace:
I20260926 19:58:51.499753 12388 rpcz_store.cc:276] 0926 19:58:49.729175 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.836110 (+1106935us) service_pool.cc:224] Handling call
0926 19:58:51.121389 (+285279us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:51.122617 (+ 1228us) catalog_manager.cc:3764] Apply alter schema
0926 19:58:51.169126 (+ 46509us) catalog_manager.cc:3796] Acquired catalog manager lock
0926 19:58:51.169368 (+ 242us) catalog_manager.cc:3954] Updating metadata on disk
0926 19:58:51.190238 (+ 20870us) catalog_manager.cc:3985] Committing alterations to in-memory state
0926 19:58:51.499437 (+309199us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:51.170546 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:51.170954 (+ 408us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:51.170978 (+ 24us) write_op.cc:435] Acquired schema lock
0926 19:58:51.170990 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:51.171214 (+ 224us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:51.171252 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:51.171273 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:51.171403 (+ 130us) tablet.cc:701] Row locks acquired
0926 19:58:51.171415 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:51.171515 (+ 100us) write_op.cc:270] Start()
0926 19:58:51.171587 (+ 72us) write_op.cc:276] Timestamp: P: 1790452731171473 usec, L: 0
0926 19:58:51.171601 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:51.172033 (+ 432us) log.cc:844] Serialized 405 byte log entry
0926 19:58:51.183600 (+ 11567us) op_driver.cc:464] REPLICATION: finished
0926 19:58:51.184326 (+ 726us) write_op.cc:301] APPLY: starting
0926 19:58:51.184373 (+ 47us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:51.184463 (+ 90us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:51.184548 (+ 85us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:51.184689 (+ 141us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:51.184811 (+ 122us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:51.184857 (+ 46us) write_op.cc:312] APPLY: finished
0926 19:58:51.185019 (+ 162us) log.cc:844] Serialized 23 byte log entry
0926 19:58:51.185155 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.185208 (+ 53us) write_op.cc:454] Released schema lock
0926 19:58:51.185251 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:51.185453 (+ 202us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.185479 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":537,"apply.run_cpu_time_us":1104,"apply.run_wall_time_us":1104,"num_ops":1,"prepare.queue_time_us":89,"prepare.run_cpu_time_us":1730,"prepare.run_wall_time_us":1730,"raft.queue_time_us":4237,"raft.run_cpu_time_us":765,"raft.run_wall_time_us":766,"replication_time_us":11906,"spinlock_wait_cycles":4480}]]}
I20260926 19:58:51.497735 13092 consensus_queue.cc:260] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:51.500140 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b39d71ccfc048848d34cf14d1f2e1fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:51.510008 13130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:58:51.511660 13130 hybrid_clock.cc:648] HybridClock initialized: now 1790452731511574 us; error 78 us; skew 500 ppm
I20260926 19:58:51.513473 13130 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:58:51.522321 13092 ts_tablet_manager.cc:1434] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.028s user 0.001s sys 0.004s
I20260926 19:58:51.523281 13092 tablet_bootstrap.cc:492] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:51.526925 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet 64149f9e4d384ddbb922291ac7de35b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:51.531874 12396 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 53) took 1072 ms (client timeout 1000 ms). Trace:
W20260926 19:58:51.532213 12396 rpcz_store.cc:269] 0926 19:58:50.459045 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.781638 (+322593us) service_pool.cc:224] Handling call
0926 19:58:51.531807 (+750169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22563,"reported_tablets":20}
I20260926 19:58:51.541692 13542 raft_consensus.cc:493] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:51.543916 13542 raft_consensus.cc:515] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.546490 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16df0828847d4e7db5702d84082fa402" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:51.547976 12696 leader_election.cc:343] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:51.550189 12387 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_df863789c3ff4a31a445becba51537cf [id=f2217cf0f71142ceb39f447b6e2cc591] alter complete (version 2)
W20260926 19:58:51.553486 12394 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 49) took 1196 ms (client timeout 1000 ms). Trace:
W20260926 19:58:51.555197 12394 rpcz_store.cc:269] 0926 19:58:50.357122 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.462832 (+105710us) service_pool.cc:224] Handling call
0926 19:58:51.553429 (+1090597us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:50.938270 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:50.938652 (+ 382us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:50.938675 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:58:50.938686 (+ 11us) tablet.cc:662] Decoding operations
0926 19:58:50.939101 (+ 415us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:50.939147 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:50.939169 (+ 22us) tablet.cc:685] Acquiring locks for 3 operations
0926 19:58:50.939423 (+ 254us) tablet.cc:701] Row locks acquired
0926 19:58:50.939435 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:50.939543 (+ 108us) write_op.cc:270] Start()
0926 19:58:50.939624 (+ 81us) write_op.cc:276] Timestamp: P: 1790452730939482 usec, L: 0
0926 19:58:50.939638 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:50.940125 (+ 487us) log.cc:844] Serialized 1302 byte log entry
0926 19:58:50.964793 (+ 24668us) op_driver.cc:464] REPLICATION: finished
0926 19:58:50.965059 (+ 266us) write_op.cc:301] APPLY: starting
0926 19:58:50.965116 (+ 57us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:50.965234 (+ 118us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:50.965332 (+ 98us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:50.965733 (+ 401us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:50.965930 (+ 197us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0926 19:58:50.965983 (+ 53us) write_op.cc:312] APPLY: finished
0926 19:58:50.966192 (+ 209us) log.cc:844] Serialized 35 byte log entry
0926 19:58:50.966351 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.966425 (+ 74us) write_op.cc:454] Released schema lock
0926 19:58:50.966456 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:50.966719 (+ 263us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:50.966743 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":103,"apply.run_cpu_time_us":1627,"apply.run_wall_time_us":1627,"num_ops":3,"prepare.queue_time_us":91,"prepare.run_cpu_time_us":2124,"prepare.run_wall_time_us":2125,"raft.queue_time_us":1362,"raft.run_cpu_time_us":714,"raft.run_wall_time_us":714,"replication_time_us":25039}]]}
I20260926 19:58:51.560421 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e337767be84507acdecfb189705bdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:51.548344 13542 leader_election.cc:290] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:51.578594 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet 886ab0feae1e49629290bc892e518816 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad76fc959e24545bdfad19f988fce93) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.445437 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 381320b4d2844e2289a5ae390e6b0503 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:51.486224 12590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:51.594614 12590 raft_consensus.cc:2468] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:51.488878 13593 tablet.cc:1751] T 0a7d5e580f5645ab95fa03d5f669dc32 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:51.509001 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 16df0828847d4e7db5702d84082fa402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:51.516443 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 64149f9e4d384ddbb922291ac7de35b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:51.553865 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16df0828847d4e7db5702d84082fa402" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:51.596855 12592 raft_consensus.cc:2468] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:51.609660 13593 tablet.cc:1751] T 99465590a0e64b35b8a8f07e8777b524 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:51.554677 13013 tablet_bootstrap.cc:700] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee: Time spent opening tablet: real 0.123s user 0.001s sys 0.000s
I20260926 19:58:51.614042 13013 tablet_bootstrap.cc:654] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:51.616643 13203 raft_consensus.cc:2272] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:51.618083 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 78cb02b71ff34f12a356248c65335d94 (DEFAULT_TABLE table=default.Table_010ba86b94b74ad18873d9aefd9c2849 [id=4234a6eb478a4e6ab0204945ec4251a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.619001 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78cb02b71ff34f12a356248c65335d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.597481 13569 ts_tablet_manager.cc:1929] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:51.620363 13569 log.cc:1199] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/0fd184e6295d47aeba809be6aab9ce1c
I20260926 19:58:51.621094 13569 ts_tablet_manager.cc:1950] T 0fd184e6295d47aeba809be6aab9ce1c P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:51.623687 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 0fd184e6295d47aeba809be6aab9ce1c (table default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]) successfully deleted
I20260926 19:58:51.613000 12734 tablet_service.cc:1511] Processing CreateTablet for tablet 78cb02b71ff34f12a356248c65335d94 (DEFAULT_TABLE table=default.Table_010ba86b94b74ad18873d9aefd9c2849 [id=4234a6eb478a4e6ab0204945ec4251a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.625344 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78cb02b71ff34f12a356248c65335d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.614531 12732 tablet_service.cc:1511] Processing CreateTablet for tablet bfdfc9dd1b6046d18459a1255be03c4c (DEFAULT_TABLE table=default.Table_5bf4644212d544c696bffa35c366b0f7 [id=cba6692a41c349d38602917980939149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.630656 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfdfc9dd1b6046d18459a1255be03c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.639920 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b25cee2337e04223b7c489d0cea2db58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:51.640200 12560 tablet_service.cc:1511] Processing CreateTablet for tablet bfdfc9dd1b6046d18459a1255be03c4c (DEFAULT_TABLE table=default.Table_5bf4644212d544c696bffa35c366b0f7 [id=cba6692a41c349d38602917980939149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.641162 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfdfc9dd1b6046d18459a1255be03c4c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:51.641381 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b25cee2337e04223b7c489d0cea2db58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:51.641387 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 3066a65ce44b4387a6f96124d9f476fb
I20260926 19:58:51.642959 12393 catalog_manager.cc:6934] ReplaceTablet: tablet 3066a65ce44b4387a6f96124d9f476fb deleted and replaced by tablet f5b4b81f11254306b56c234f22b86ea5
I20260926 19:58:51.645133 12736 tablet_service.cc:1558] Processing DeleteTablet for tablet 3066a65ce44b4387a6f96124d9f476fb with delete_type TABLET_DATA_DELETED (replaced by tablet f5b4b81f11254306b56c234f22b86ea5) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.647845 12736 tablet_service.cc:1511] Processing CreateTablet for tablet f3af0a1fb53b4181a06965845d585f7f (DEFAULT_TABLE table=default.Table_a14b58f3fb9842d6a5ab2711958ddec9 [id=f475fa17f24c44a6b881bf347e147b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.600453 13012 tablet_bootstrap.cc:492] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:51.648777 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3af0a1fb53b4181a06965845d585f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.648980 13012 ts_tablet_manager.cc:1403] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.331s user 0.011s sys 0.003s
I20260926 19:58:51.651345 13012 raft_consensus.cc:359] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.651949 13012 raft_consensus.cc:385] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:51.652196 13012 raft_consensus.cc:740] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:51.652806 13012 consensus_queue.cc:260] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.655313 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 3066a65ce44b4387a6f96124d9f476fb with delete_type TABLET_DATA_DELETED (replaced by tablet f5b4b81f11254306b56c234f22b86ea5) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:51.656113 12561 tablet_service.cc:1511] Processing CreateTablet for tablet f3af0a1fb53b4181a06965845d585f7f (DEFAULT_TABLE table=default.Table_a14b58f3fb9842d6a5ab2711958ddec9 [id=f475fa17f24c44a6b881bf347e147b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.657063 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3af0a1fb53b4181a06965845d585f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.664269 13203 ts_tablet_manager.cc:1916] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:51.666846 12921 raft_consensus.cc:3060] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:51.667922 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_408479a582a04c8b829f7474e9cd00ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:51.669296 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_408479a582a04c8b829f7474e9cd00ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:51.672710 12695 leader_election.cc:304] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:51.676424 13092 tablet_bootstrap.cc:654] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:51.686409 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 5d864d73048548baa056c60a58eb12ff (DEFAULT_TABLE table=default.Table_9d1e6cd14bd8461faff475b85d9b9368 [id=6619498bab1e4399a9c76c242a005963]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.686883 12741 tablet_service.cc:1511] Processing CreateTablet for tablet cb5e54722cfe43e5ada9feafaaa23964 (DEFAULT_TABLE table=default.Table_15cc720c12e6419d8ca44d8485d7c80b [id=b0d4ae3113fd4476a6e1fc32215ba752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.687772 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d864d73048548baa056c60a58eb12ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.693835 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb5e54722cfe43e5ada9feafaaa23964. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:51.704602 13640 log.cc:927] Time spent T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5: Append to log took a long time: real 0.127s user 0.001s sys 0.000s
I20260926 19:58:51.708552 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 5d864d73048548baa056c60a58eb12ff (DEFAULT_TABLE table=default.Table_9d1e6cd14bd8461faff475b85d9b9368 [id=6619498bab1e4399a9c76c242a005963]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.709591 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d864d73048548baa056c60a58eb12ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.718535 13569 tablet_replica.cc:333] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:51.718961 13013 tablet_bootstrap.cc:492] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:51.719530 13569 raft_consensus.cc:2243] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:51.729671 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 5450b4cd35f04904845f1612fe58be52 (DEFAULT_TABLE table=default.Table_ae3912e6fea44f88a074a4bbed57b0cd [id=006cff92a43c4e07a583301fb50dad29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.730732 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5450b4cd35f04904845f1612fe58be52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.731397 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 3df392d277a447aabf4320a7d9a43880 (DEFAULT_TABLE table=default.Table_b294fbdc488a45a3b54ff6e45272ef72 [id=f4a644d228fe4e47985e6142a522275d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.733618 13012 ts_tablet_manager.cc:1434] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.084s user 0.003s sys 0.003s
I20260926 19:58:51.733685 12389 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 6e20b1a22af944a0aad0fee5fc835f6e
I20260926 19:58:51.736213 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 4d153e60ae5d486cb91c2cc623b99985 (DEFAULT_TABLE table=default.Table_85695893a5b24687959b83a8360d1443 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.740329 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3df392d277a447aabf4320a7d9a43880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.746348 12735 tablet_service.cc:1558] Processing DeleteTablet for tablet 6e20b1a22af944a0aad0fee5fc835f6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:51.748595 13585 tablet.cc:1751] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260926 19:58:51.749981 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 77405.7us
I20260926 19:58:51.750710 12695 leader_election.cc:304] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:51.751087 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_174debbe544e47ccbcf8faef177fdd0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:51.752499 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_174debbe544e47ccbcf8faef177fdd0b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:51.757628 13013 ts_tablet_manager.cc:1403] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.593s user 0.010s sys 0.000s
I20260926 19:58:51.760366 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_35146d1cd4df4da594f520e22409c68f" } modify_external_catalogs: true
I20260926 19:58:51.760254 13013 raft_consensus.cc:359] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.760847 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_35146d1cd4df4da594f520e22409c68f" } modify_external_catalogs: true
I20260926 19:58:51.760957 13013 raft_consensus.cc:385] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:51.768256 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_cb7930cd5c56467d9d6ccf644263f999" } new_table_name: "default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7" modify_external_catalogs: true
I20260926 19:58:51.771062 12392 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_0f5f1b1ec9ab499a9419665ce344105e" } new_table_name: "default.Table_43587029a971483b88871b746dc9088c" modify_external_catalogs: true
I20260926 19:58:51.774389 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_91489cb29847444d9e79a0587e38a517"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:51.775837 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91489cb29847444d9e79a0587e38a517 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:51.781023 12380 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 3df392d277a447aabf4320a7d9a43880 (table default.Table_b294fbdc488a45a3b54ff6e45272ef72 [id=f4a644d228fe4e47985e6142a522275d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3df392d277a447aabf4320a7d9a43880 consensus state has no leader
I20260926 19:58:51.805109 13629 raft_consensus.cc:493] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:51.805605 13629 raft_consensus.cc:515] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.807375 13629 leader_election.cc:290] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:51.808290 13403 ts_tablet_manager.cc:1929] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:51.813925 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 3df392d277a447aabf4320a7d9a43880 (DEFAULT_TABLE table=default.Table_b294fbdc488a45a3b54ff6e45272ef72 [id=f4a644d228fe4e47985e6142a522275d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.815088 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3df392d277a447aabf4320a7d9a43880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.822958 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 5450b4cd35f04904845f1612fe58be52 (DEFAULT_TABLE table=default.Table_ae3912e6fea44f88a074a4bbed57b0cd [id=006cff92a43c4e07a583301fb50dad29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.823395 12558 tablet_service.cc:1511] Processing CreateTablet for tablet 4d153e60ae5d486cb91c2cc623b99985 (DEFAULT_TABLE table=default.Table_85695893a5b24687959b83a8360d1443 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.824134 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5450b4cd35f04904845f1612fe58be52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.824690 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d153e60ae5d486cb91c2cc623b99985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.761250 13013 raft_consensus.cc:740] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:51.835866 13013 consensus_queue.cc:260] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.855504 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:51.856148 12592 raft_consensus.cc:2468] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:51.865481 13092 tablet_bootstrap.cc:492] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:51.866014 13092 ts_tablet_manager.cc:1403] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.343s user 0.010s sys 0.000s
I20260926 19:58:51.866676 12390 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet cab58d9ba156436ab50f2ae9fea8f3f4
I20260926 19:58:51.867214 12858 leader_election.cc:304] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:51.867569 12564 tablet_service.cc:1511] Processing CreateTablet for tablet d4434cb08d0f41fd96283de49c53b3e8 (DEFAULT_TABLE table=default.Table_447c2fd2552a48d0897da03fd0d1d963 [id=fcf0c9e6dcd647d09ad31bbc20abd068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.870612 13012 tablet_bootstrap.cc:492] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:51.873446 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 122810us
I20260926 19:58:51.875192 13130 webserver.cc:533] Webserver started at http://127.11.233.253:36963/ using document root <none> and password file <none>
I20260926 19:58:51.876793 13130 fs_manager.cc:362] Metadata directory not provided
I20260926 19:58:51.877104 13130 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:58:51.886859 13013 ts_tablet_manager.cc:1434] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.129s user 0.005s sys 0.000s
I20260926 19:58:51.808604 13403 log.cc:1199] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/1455b99a65ff44ff933825196708f485
I20260926 19:58:51.890997 13403 ts_tablet_manager.cc:1950] T 1455b99a65ff44ff933825196708f485 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:51.897876 13629 raft_consensus.cc:2804] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:51.898255 13629 raft_consensus.cc:493] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:51.898541 13629 raft_consensus.cc:3060] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:51.906706 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 78cb02b71ff34f12a356248c65335d94 (DEFAULT_TABLE table=default.Table_010ba86b94b74ad18873d9aefd9c2849 [id=4234a6eb478a4e6ab0204945ec4251a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.907680 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78cb02b71ff34f12a356248c65335d94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.914503 12900 tablet_service.cc:1511] Processing CreateTablet for tablet bfdfc9dd1b6046d18459a1255be03c4c (DEFAULT_TABLE table=default.Table_5bf4644212d544c696bffa35c366b0f7 [id=cba6692a41c349d38602917980939149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.915464 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfdfc9dd1b6046d18459a1255be03c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.918381 13092 raft_consensus.cc:359] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.919196 13092 raft_consensus.cc:385] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:51.920238 13092 raft_consensus.cc:740] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:51.921257 13013 tablet_bootstrap.cc:492] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:51.920943 13092 consensus_queue.cc:260] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.925956 13092 ts_tablet_manager.cc:1434] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.060s user 0.006s sys 0.001s
I20260926 19:58:51.928046 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 201) took 1050 ms. Trace:
I20260926 19:58:51.928550 13403 rpcz_store.cc:276] 0926 19:58:50.877865 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.881876 (+ 4011us) service_pool.cc:224] Handling call
0926 19:58:51.084594 (+202718us) ts_tablet_manager.cc:1151] Deleting tablet 1455b99a65ff44ff933825196708f485
0926 19:58:51.084638 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:51.660869 (+576231us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:51.808171 (+147302us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:51.927990 (+119819us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34602,"tablet-delete.queue_time_us":202150}
I20260926 19:58:51.905328 13629 raft_consensus.cc:515] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.929957 13403 tablet_replica.cc:333] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:51.930639 13403 raft_consensus.cc:2243] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:51.930797 13092 tablet_bootstrap.cc:492] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:51.931591 13403 raft_consensus.cc:2272] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:51.932114 13629 leader_election.cc:290] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:51.933066 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ff071a16f1244cebc8c93e5508280a3 consensus state has no leader
I20260926 19:58:51.942370 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet 3066a65ce44b4387a6f96124d9f476fb with delete_type TABLET_DATA_DELETED (replaced by tablet f5b4b81f11254306b56c234f22b86ea5) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:51.943167 12897 tablet_service.cc:1511] Processing CreateTablet for tablet f3af0a1fb53b4181a06965845d585f7f (DEFAULT_TABLE table=default.Table_a14b58f3fb9842d6a5ab2711958ddec9 [id=f475fa17f24c44a6b881bf347e147b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.944254 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3af0a1fb53b4181a06965845d585f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.945019 13092 tablet_bootstrap.cc:654] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:51.951155 12392 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ae465ba0677434590c0732a654042b5 has no consensus state
I20260926 19:58:51.953559 13549 raft_consensus.cc:493] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:51.954115 13549 raft_consensus.cc:515] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.956033 13092 tablet_bootstrap.cc:492] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:51.956512 13092 ts_tablet_manager.cc:1403] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.004s
I20260926 19:58:51.962915 13092 raft_consensus.cc:359] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.964587 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b70cb5834613437fab92972194f769da" } modify_external_catalogs: true
I20260926 19:58:51.965101 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b70cb5834613437fab92972194f769da" } modify_external_catalogs: true
I20260926 19:58:51.965837 13092 raft_consensus.cc:385] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:51.966151 13092 raft_consensus.cc:740] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:51.967947 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet 6e20b1a22af944a0aad0fee5fc835f6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:51.967650 13092 consensus_queue.cc:260] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:51.969017 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 71065b80482442699f4b207cb8f9d40c (DEFAULT_TABLE table=default.Table_861452498c1d4bcab2471cd4e97d830a [id=b78e71ba6629447f897f414153987f30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.975268 12898 tablet_service.cc:1511] Processing CreateTablet for tablet cb5e54722cfe43e5ada9feafaaa23964 (DEFAULT_TABLE table=default.Table_15cc720c12e6419d8ca44d8485d7c80b [id=b0d4ae3113fd4476a6e1fc32215ba752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.976255 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb5e54722cfe43e5ada9feafaaa23964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.977288 12391 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_979fb16236da4673983d947c83506fd3 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 2 got 1
I20260926 19:58:51.979797 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4434cb08d0f41fd96283de49c53b3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.980465 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71065b80482442699f4b207cb8f9d40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.980681 12391 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:51.982133 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 5d864d73048548baa056c60a58eb12ff (DEFAULT_TABLE table=default.Table_9d1e6cd14bd8461faff475b85d9b9368 [id=6619498bab1e4399a9c76c242a005963]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.983053 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d864d73048548baa056c60a58eb12ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.983316 12560 tablet_service.cc:1511] Processing CreateTablet for tablet cb5e54722cfe43e5ada9feafaaa23964 (DEFAULT_TABLE table=default.Table_15cc720c12e6419d8ca44d8485d7c80b [id=b0d4ae3113fd4476a6e1fc32215ba752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.984220 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb5e54722cfe43e5ada9feafaaa23964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.976284 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 3df392d277a447aabf4320a7d9a43880 (DEFAULT_TABLE table=default.Table_b294fbdc488a45a3b54ff6e45272ef72 [id=f4a644d228fe4e47985e6142a522275d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.986820 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3df392d277a447aabf4320a7d9a43880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.996806 12902 tablet_service.cc:1558] Processing DeleteTablet for tablet 6e20b1a22af944a0aad0fee5fc835f6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:51.996989 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 4d153e60ae5d486cb91c2cc623b99985 (DEFAULT_TABLE table=default.Table_85695893a5b24687959b83a8360d1443 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:51.997960 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d153e60ae5d486cb91c2cc623b99985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:51.998070 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 1455b99a65ff44ff933825196708f485 (table default.Table_f1e412eb951f446990776744666978fc [id=bfb91cbd91534b4699fd4423b9923881]) successfully deleted
I20260926 19:58:51.999918 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:52.000433 12592 raft_consensus.cc:3060] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.000579 12896 tablet_service.cc:1511] Processing CreateTablet for tablet d4434cb08d0f41fd96283de49c53b3e8 (DEFAULT_TABLE table=default.Table_447c2fd2552a48d0897da03fd0d1d963 [id=fcf0c9e6dcd647d09ad31bbc20abd068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.001648 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4434cb08d0f41fd96283de49c53b3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.001839 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:52.002653 13569 raft_consensus.cc:2272] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:52.006342 12895 tablet_service.cc:1511] Processing CreateTablet for tablet 71065b80482442699f4b207cb8f9d40c (DEFAULT_TABLE table=default.Table_861452498c1d4bcab2471cd4e97d830a [id=b78e71ba6629447f897f414153987f30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.007023 12592 raft_consensus.cc:2468] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:52.007416 12895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71065b80482442699f4b207cb8f9d40c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:52.009413 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 135293us
I20260926 19:58:52.009923 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d153e60ae5d486cb91c2cc623b99985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.011147 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 71065b80482442699f4b207cb8f9d40c (DEFAULT_TABLE table=default.Table_861452498c1d4bcab2471cd4e97d830a [id=b78e71ba6629447f897f414153987f30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.016031 13549 leader_election.cc:290] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:52.016923 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b319b685344a848462b78617eb6c22" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:52.017530 12921 raft_consensus.cc:2468] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:52.018781 12521 leader_election.cc:304] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:52.019954 13596 raft_consensus.cc:2804] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:52.020339 13596 raft_consensus.cc:493] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:52.020637 13596 raft_consensus.cc:3060] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.029241 13599 raft_consensus.cc:2804] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:52.029624 13599 raft_consensus.cc:493] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:52.029910 13599 raft_consensus.cc:3060] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.029224 12391 catalog_manager.cc:5838] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:52.034019 12391 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:52.036782 13598 raft_consensus.cc:493] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:52.037190 13598 raft_consensus.cc:515] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.039768 13598 leader_election.cc:290] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:52.025851 12735 tablet_service.cc:1511] Processing CreateTablet for tablet d4434cb08d0f41fd96283de49c53b3e8 (DEFAULT_TABLE table=default.Table_447c2fd2552a48d0897da03fd0d1d963 [id=fcf0c9e6dcd647d09ad31bbc20abd068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.042256 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 5450b4cd35f04904845f1612fe58be52 (DEFAULT_TABLE table=default.Table_ae3912e6fea44f88a074a4bbed57b0cd [id=006cff92a43c4e07a583301fb50dad29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.043200 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5450b4cd35f04904845f1612fe58be52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.044392 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b319b685344a848462b78617eb6c22" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:52.045001 12921 raft_consensus.cc:2468] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:51.973672 13092 ts_tablet_manager.cc:1434] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.017s user 0.009s sys 0.000s
I20260926 19:58:52.052640 13092 tablet_bootstrap.cc:492] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:52.053844 12858 leader_election.cc:304] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:52.054484 13629 raft_consensus.cc:2804] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:52.054903 13629 raft_consensus.cc:697] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:52.054155 12391 catalog_manager.cc:5838] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:52.055588 12391 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:52.055660 13629 consensus_queue.cc:237] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.063962 13092 tablet_bootstrap.cc:654] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.047473 12696 leader_election.cc:304] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:52.075475 13569 ts_tablet_manager.cc:1916] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:52.075701 13092 tablet_bootstrap.cc:492] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:52.076061 13092 ts_tablet_manager.cc:1403] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.000s
I20260926 19:58:52.078015 13012 tablet_bootstrap.cc:654] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:51.933285 13403 ts_tablet_manager.cc:1916] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:52.078316 13092 raft_consensus.cc:359] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.081141 12395 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_ae3912e6fea44f88a074a4bbed57b0cd" } modify_external_catalogs: true
I20260926 19:58:52.081805 12395 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_ae3912e6fea44f88a074a4bbed57b0cd" } modify_external_catalogs: true
I20260926 19:58:52.082697 13612 raft_consensus.cc:2804] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:52.083037 13612 raft_consensus.cc:493] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:52.083343 13612 raft_consensus.cc:3060] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.078888 13092 raft_consensus.cc:385] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.085189 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_8e95dc18c4dd485bb6eae37e9dccaf1f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:52.085769 13092 raft_consensus.cc:740] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.086412 13092 consensus_queue.cc:260] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:52.086758 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e95dc18c4dd485bb6eae37e9dccaf1f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.088792 13092 ts_tablet_manager.cc:1434] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260926 19:58:52.089641 13092 tablet_bootstrap.cc:492] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:52.091131 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_aa7b619547ec4df09f564d512fb394c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.092962 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa7b619547ec4df09f564d512fb394c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.094172 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9f2175a4adac4b7f84b726e72d7aaddb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:52.068301 13013 tablet_bootstrap.cc:654] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:52.096027 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f2175a4adac4b7f84b726e72d7aaddb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.100520 13092 tablet_bootstrap.cc:654] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.103345 13203 ts_tablet_manager.cc:1929] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:52.103736 13203 log.cc:1199] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/3215bbd5be56438491a73925ac9ccbe6
I20260926 19:58:52.121762 13590 tablet.cc:1751] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:52.126899 13549 raft_consensus.cc:2804] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:52.137130 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_13a9930424e74b249d88b4f42195f0ad" } new_table_name: "default.Table_bfdeb281a21842bc9cf53aafa87f80cd" modify_external_catalogs: true
I20260926 19:58:52.143775 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b319b685344a848462b78617eb6c22" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:52.144955 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:52.147243 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71065b80482442699f4b207cb8f9d40c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.151409 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4434cb08d0f41fd96283de49c53b3e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.151684 12390 master_service.cc:960] ReplaceTablet: received request to replace tablet 56fd4c88f9304f19883c3e5da8f937b5 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:52.154810 13596 raft_consensus.cc:515] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:52.158936 12382 consensus_peers.cc:597] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 -> Peer db2490adb6e84090b780f52470e580ab (127.11.233.253:37913): Couldn't send request to peer db2490adb6e84090b780f52470e580ab. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.253:37913: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260926 19:58:52.162683 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_01a9ebf62e394638933953cc6654b838"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.164475 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01a9ebf62e394638933953cc6654b838 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.168059 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16df0828847d4e7db5702d84082fa402" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:52.168324 13203 ts_tablet_manager.cc:1950] T 3215bbd5be56438491a73925ac9ccbe6 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:52.169095 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 211) took 1281 ms. Trace:
W20260926 19:58:52.174499 12696 leader_election.cc:343] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:52.177335 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_ee77d6a00998425b94479cf721435bf0" } new_table_name: "default.Table_abeab641cddf49ca82c394e6e2072af4" modify_external_catalogs: true
I20260926 19:58:52.182591 13599 raft_consensus.cc:515] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.183887 12757 raft_consensus.cc:2393] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:52.169816 13203 rpcz_store.cc:276] 0926 19:58:50.887429 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.887542 (+ 113us) service_pool.cc:224] Handling call
0926 19:58:51.420593 (+533051us) ts_tablet_manager.cc:1151] Deleting tablet 3215bbd5be56438491a73925ac9ccbe6
0926 19:58:51.420647 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:51.978512 (+557865us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.103220 (+124708us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.169042 (+ 65822us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":484191}
I20260926 19:58:52.195564 13596 leader_election.cc:290] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:52.194567 13203 tablet_replica.cc:333] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:52.200760 13203 raft_consensus.cc:2243] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:52.201225 13203 raft_consensus.cc:2272] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:52.214207 13203 ts_tablet_manager.cc:1916] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:52.220824 13599 leader_election.cc:290] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:52.223666 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:52.229247 13585 tablet.cc:1751] T 2411a153e7194406a2bb7b879c08d7dd P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:52.236501 13403 ts_tablet_manager.cc:1929] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:52.241724 13403 log.cc:1199] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/cd92a1a62ab24647bef7c9eefdf9c9d8
I20260926 19:58:52.243322 13596 raft_consensus.cc:493] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:52.245015 13596 raft_consensus.cc:515] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.243523 13403 ts_tablet_manager.cc:1950] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:52.256886 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 200) took 1379 ms. Trace:
I20260926 19:58:52.258814 13612 raft_consensus.cc:515] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.260111 12759 raft_consensus.cc:2393] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f6e9d6439fd54b28925a68a6145d3dee in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:52.265650 13403 rpcz_store.cc:276] 0926 19:58:50.877493 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.896151 (+ 18658us) service_pool.cc:224] Handling call
0926 19:58:51.929816 (+1033665us) ts_tablet_manager.cc:1151] Deleting tablet cd92a1a62ab24647bef7c9eefdf9c9d8
0926 19:58:51.929859 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:52.153353 (+223494us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.236380 (+ 83027us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.256826 (+ 20446us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1032913}
I20260926 19:58:52.268517 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet cd92a1a62ab24647bef7c9eefdf9c9d8 (table default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]) successfully deleted
I20260926 19:58:52.270157 13403 tablet_replica.cc:333] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:52.276245 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 56fd4c88f9304f19883c3e5da8f937b5
I20260926 19:58:52.279070 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 9599f427a64f40d7b185b23a849b46f3) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:52.279232 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 9599f427a64f40d7b185b23a849b46f3) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:52.279623 12390 catalog_manager.cc:6934] ReplaceTablet: tablet 56fd4c88f9304f19883c3e5da8f937b5 deleted and replaced by tablet 9599f427a64f40d7b185b23a849b46f3
I20260926 19:58:52.127264 13549 raft_consensus.cc:493] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:52.291567 13549 raft_consensus.cc:3060] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.295251 13012 tablet_bootstrap.cc:492] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:52.295691 13012 ts_tablet_manager.cc:1403] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.504s user 0.013s sys 0.001s
I20260926 19:58:52.296489 13403 raft_consensus.cc:2243] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:52.298096 13403 raft_consensus.cc:2272] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:52.298102 13012 raft_consensus.cc:359] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.298702 13012 raft_consensus.cc:385] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.301043 13403 ts_tablet_manager.cc:1916] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:52.301342 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59d1955b12dd47899b8a226fe0e980ae consensus state has no leader
I20260926 19:58:52.302641 13596 leader_election.cc:290] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:52.305285 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16897e7489e9445ba5389a4f3507a739" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:52.307130 12696 leader_election.cc:343] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:52.311241 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b319b685344a848462b78617eb6c22" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:52.311769 12921 raft_consensus.cc:3060] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:52.320699 12519 consensus_peers.cc:597] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:52.322602 12924 raft_consensus.cc:3060] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.339802 13013 tablet_bootstrap.cc:492] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:52.340273 13013 ts_tablet_manager.cc:1403] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.420s user 0.030s sys 0.006s
I20260926 19:58:52.349066 13593 tablet.cc:1751] T 2411a153e7194406a2bb7b879c08d7dd P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:52.353637 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b319b685344a848462b78617eb6c22" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:52.355718 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16df0828847d4e7db5702d84082fa402" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:52.356257 12589 raft_consensus.cc:3060] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.362329 12590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5324b1b682f43d4a689b6fafff3df01" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:52.363428 12590 raft_consensus.cc:2393] T a5324b1b682f43d4a689b6fafff3df01 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in this term.
I20260926 19:58:52.364432 12589 raft_consensus.cc:2468] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:52.342676 13013 raft_consensus.cc:359] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.365912 13013 raft_consensus.cc:385] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.366657 13013 raft_consensus.cc:740] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.368470 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 3215bbd5be56438491a73925ac9ccbe6 (table default.Table_2dd176d225964a3c827572433ed19d99 [id=9e27cd26087444739cb87ca72eb203ba]) successfully deleted
I20260926 19:58:52.366580 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16897e7489e9445ba5389a4f3507a739" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:52.370278 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b319b685344a848462b78617eb6c22" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:52.375257 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_de6c58034b1949dfa5e5f83bd07db6d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.376727 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de6c58034b1949dfa5e5f83bd07db6d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.377111 13013 consensus_queue.cc:260] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.377650 13660 raft_consensus.cc:493] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:52.381747 13660 raft_consensus.cc:515] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.382304 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_66bb1825f60742cc90be30615c83724a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:52.383565 13660 leader_election.cc:290] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:52.383738 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_66bb1825f60742cc90be30615c83724a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.384582 13013 ts_tablet_manager.cc:1434] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260926 19:58:52.385450 13013 tablet_bootstrap.cc:492] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:52.385268 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff071a16f1244cebc8c93e5508280a3" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:52.388461 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff071a16f1244cebc8c93e5508280a3" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:52.389859 12521 leader_election.cc:343] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:52.393846 13403 ts_tablet_manager.cc:1929] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:52.394199 13403 log.cc:1199] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/886ab0feae1e49629290bc892e518816
I20260926 19:58:52.394810 13403 ts_tablet_manager.cc:1950] T 886ab0feae1e49629290bc892e518816 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:52.394850 13598 consensus_queue.cc:1048] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:52.396000 13092 tablet_bootstrap.cc:492] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:52.396570 13092 ts_tablet_manager.cc:1403] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.307s user 0.049s sys 0.078s
W20260926 19:58:52.397132 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 54720.3us
I20260926 19:58:52.398938 13092 raft_consensus.cc:359] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.399547 13092 raft_consensus.cc:385] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.399802 13092 raft_consensus.cc:740] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.400385 13092 consensus_queue.cc:260] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.402576 13092 ts_tablet_manager.cc:1434] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 19:58:52.403353 13092 tablet_bootstrap.cc:492] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:52.406635 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 204) took 1494 ms. Trace:
I20260926 19:58:52.412371 13092 tablet_bootstrap.cc:654] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.412570 13013 tablet_bootstrap.cc:654] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.414170 12695 leader_election.cc:304] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:52.406932 13403 rpcz_store.cc:276] 0926 19:58:50.912737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.914234 (+ 1497us) service_pool.cc:224] Handling call
0926 19:58:52.269615 (+1355381us) ts_tablet_manager.cc:1151] Deleting tablet 886ab0feae1e49629290bc892e518816
0926 19:58:52.269657 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:52.335202 (+ 65545us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.393730 (+ 58528us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.406582 (+ 12852us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1354562}
I20260926 19:58:52.415287 13617 consensus_queue.cc:1048] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:52.415486 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 886ab0feae1e49629290bc892e518816 (table default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189]) successfully deleted
I20260926 19:58:52.416311 13403 tablet_replica.cc:333] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:52.416421 13617 raft_consensus.cc:2749] T a5324b1b682f43d4a689b6fafff3df01 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:52.417140 13403 raft_consensus.cc:2243] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:52.417629 13403 raft_consensus.cc:2272] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:52.418602 12695 leader_election.cc:304] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:52.419288 13587 raft_consensus.cc:2804] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:52.419359 13403 ts_tablet_manager.cc:1916] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:52.419777 13587 raft_consensus.cc:697] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:52.420550 13587 consensus_queue.cc:237] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.430471 13092 tablet_bootstrap.cc:492] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:52.430562 13612 leader_election.cc:290] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:52.430894 13092 ts_tablet_manager.cc:1403] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.004s
I20260926 19:58:52.435880 13092 raft_consensus.cc:359] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.436497 13092 raft_consensus.cc:385] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.436779 13092 raft_consensus.cc:740] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.437438 13092 consensus_queue.cc:260] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.439249 13013 tablet_bootstrap.cc:492] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:52.439630 13013 ts_tablet_manager.cc:1403] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.054s user 0.012s sys 0.000s
I20260926 19:58:52.440238 13092 ts_tablet_manager.cc:1434] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260926 19:58:52.441080 13092 tablet_bootstrap.cc:492] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:52.446074 13013 raft_consensus.cc:359] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.446714 13013 raft_consensus.cc:385] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.446959 13013 raft_consensus.cc:740] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.447537 13013 consensus_queue.cc:260] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.448508 13629 raft_consensus.cc:493] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:52.448935 13629 raft_consensus.cc:515] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.450749 13629 leader_election.cc:290] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:52.452450 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d11cb23144691809c554c125cad0e" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:52.453148 12589 raft_consensus.cc:2393] T 552d11cb23144691809c554c125cad0e P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:52.454315 13130 fs_manager.cc:714] Time spent opening directory manager: real 0.431s user 0.006s sys 0.006s
I20260926 19:58:52.455478 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "552d11cb23144691809c554c125cad0e" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:52.456101 13013 ts_tablet_manager.cc:1434] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260926 19:58:52.457015 13013 tablet_bootstrap.cc:492] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:52.456947 12857 leader_election.cc:304] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:52.457888 13629 raft_consensus.cc:3060] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.464365 13629 raft_consensus.cc:2749] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:52.470276 13598 consensus_queue.cc:1048] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:52.474710 12921 raft_consensus.cc:2468] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:52.475610 13403 ts_tablet_manager.cc:1929] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:52.475940 13403 log.cc:1199] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/381320b4d2844e2289a5ae390e6b0503
I20260926 19:58:52.476693 13403 ts_tablet_manager.cc:1950] T 381320b4d2844e2289a5ae390e6b0503 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:52.476841 12696 leader_election.cc:304] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:52.477663 13598 raft_consensus.cc:2804] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:52.478463 13598 raft_consensus.cc:697] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:52.479013 13403 tablet_replica.cc:333] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:52.479182 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 381320b4d2844e2289a5ae390e6b0503 (table default.Table_a8870fb0212e43b788b3fced198aed4e [id=2206b9a20656421f8bbd96810835c982]) successfully deleted
I20260926 19:58:52.478170 13092 tablet_bootstrap.cc:654] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.479353 13598 consensus_queue.cc:237] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.487041 13013 tablet_bootstrap.cc:654] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.491436 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_27e7d0d0005441c8b6d63013d60972aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.493373 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27e7d0d0005441c8b6d63013d60972aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.298969 13012 raft_consensus.cc:740] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.497174 13012 consensus_queue.cc:260] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.498047 12759 raft_consensus.cc:2468] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:52.498402 13403 raft_consensus.cc:2243] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:52.498901 13403 raft_consensus.cc:2272] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:52.499465 13629 raft_consensus.cc:493] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:52.499653 12519 leader_election.cc:304] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:52.499902 13629 raft_consensus.cc:515] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.500653 13662 raft_consensus.cc:2804] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:52.500950 13012 ts_tablet_manager.cc:1434] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.205s user 0.005s sys 0.000s
I20260926 19:58:52.501375 13662 raft_consensus.cc:493] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:52.501693 13662 raft_consensus.cc:3060] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.501820 13012 tablet_bootstrap.cc:492] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:52.502789 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c8916d88dd48998eab514bd3c538b9" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:52.510577 12857 leader_election.cc:343] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:52.511221 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9f2175a4adac4b7f84b726e72d7aaddb" } new_table_name: "default.Table_9cad47f6e7924243acf6f9c6b9a0573f" modify_external_catalogs: true
I20260926 19:58:52.512123 13668 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:58:52.514271 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c8916d88dd48998eab514bd3c538b9" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:52.514837 12589 raft_consensus.cc:2468] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:52.516044 12858 leader_election.cc:304] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:52.516820 13633 raft_consensus.cc:2804] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:52.517122 13633 raft_consensus.cc:493] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:52.517418 13633 raft_consensus.cc:3060] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.520992 13013 tablet_bootstrap.cc:492] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:52.521374 13013 ts_tablet_manager.cc:1403] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.065s user 0.010s sys 0.000s
I20260926 19:58:52.521993 13629 leader_election.cc:290] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:52.532279 13403 ts_tablet_manager.cc:1916] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:52.537588 13013 raft_consensus.cc:359] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.538180 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 9599f427a64f40d7b185b23a849b46f3) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:52.538277 13013 raft_consensus.cc:385] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.538522 13013 raft_consensus.cc:740] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.539083 13662 raft_consensus.cc:515] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.539804 13013 consensus_queue.cc:260] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:52.540242 12695 leader_election.cc:343] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:52.541558 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff071a16f1244cebc8c93e5508280a3" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:52.540910 12695 leader_election.cc:304] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:52.543571 13587 raft_consensus.cc:2749] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:52.543578 12521 leader_election.cc:343] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:52.545351 13013 ts_tablet_manager.cc:1434] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
W20260926 19:58:52.545722 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4ee22184d1240a2b62bcdddb596c985 has no consensus state
I20260926 19:58:52.549767 13633 raft_consensus.cc:515] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.552124 13130 fs_manager.cc:730] Time spent opening block manager: real 0.093s user 0.007s sys 0.000s
I20260926 19:58:52.554762 13013 tablet_bootstrap.cc:492] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:52.552591 13130 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
uuid: "db2490adb6e84090b780f52470e580ab"
format_stamp: "Formatted at 2026-09-26 19:58:23 on dist-test-slave-jcdj"
I20260926 19:58:52.563441 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c8916d88dd48998eab514bd3c538b9" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:52.564013 12593 raft_consensus.cc:3060] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.565856 13662 leader_election.cc:290] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:52.565582 13633 leader_election.cc:290] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:52.566835 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff071a16f1244cebc8c93e5508280a3" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:52.567371 12759 raft_consensus.cc:3060] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.568516 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c8916d88dd48998eab514bd3c538b9" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
W20260926 19:58:52.570607 12857 leader_election.cc:343] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:52.579077 13092 tablet_bootstrap.cc:492] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:52.579486 13092 ts_tablet_manager.cc:1403] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.139s user 0.005s sys 0.006s
I20260926 19:58:52.585556 12593 raft_consensus.cc:2468] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:52.529290 13549 raft_consensus.cc:515] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.589794 12858 leader_election.cc:304] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:52.590778 13629 raft_consensus.cc:2804] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:52.591202 13629 raft_consensus.cc:697] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:52.591917 13629 consensus_queue.cc:237] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.594067 12594 raft_consensus.cc:2393] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:52.598910 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f049d03e319247ad9be123472530cea4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.600358 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f049d03e319247ad9be123472530cea4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.596220 13092 raft_consensus.cc:359] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.606065 13092 raft_consensus.cc:385] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.606356 13092 raft_consensus.cc:740] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.607059 13092 consensus_queue.cc:260] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.611359 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 51f5d8149d474ab1b52d53fc0b490032
I20260926 19:58:52.611840 13092 ts_tablet_manager.cc:1434] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260926 19:58:52.612867 12901 tablet_service.cc:1558] Processing DeleteTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:52.615170 13092 tablet_bootstrap.cc:492] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:52.617925 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b319b685344a848462b78617eb6c22" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:52.618907 12924 raft_consensus.cc:2393] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f6e9d6439fd54b28925a68a6145d3dee in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:52.621284 13092 tablet_bootstrap.cc:654] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.621349 13013 tablet_bootstrap.cc:654] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.633308 12393 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 2411a153e7194406a2bb7b879c08d7dd (table default.Table_979fb16236da4673983d947c83506fd3 [id=02441df5c4e6405fa8a72aa72b16687e]). Expected version 2 got 1
I20260926 19:58:52.634954 12393 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:52.635795 12393 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:52.636446 12393 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 99465590a0e64b35b8a8f07e8777b524 (table default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3]). Expected version 2 got 0
I20260926 19:58:52.638185 12451 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 51f5d8149d474ab1b52d53fc0b490032
I20260926 19:58:52.638293 13549 leader_election.cc:290] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:52.556877 13130 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:58:52.647014 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:52.647631 12898 tablet_service.cc:1511] Processing CreateTablet for tablet b58bb831147e44d496fea1f83fd0504c (DEFAULT_TABLE table=default.Table_8b39d71ccfc048848d34cf14d1f2e1fd [id=96e6a79e95024055a1cfa851677bd080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.649807 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58bb831147e44d496fea1f83fd0504c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.650825 12389 catalog_manager.cc:5838] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195). New cstate: current_term: 1 leader_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:52.655059 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:52.653645 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ba5394bd03c6428eb734bb5133243b18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.656807 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba5394bd03c6428eb734bb5133243b18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.657572 12395 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 5450b4cd35f04904845f1612fe58be52
I20260926 19:58:52.657599 13629 raft_consensus.cc:493] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:52.658018 13629 raft_consensus.cc:515] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.660938 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff071a16f1244cebc8c93e5508280a3" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:52.661618 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 5450b4cd35f04904845f1612fe58be52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:52.661898 12592 raft_consensus.cc:2393] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:52.652537 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9220288c84b847fea7df2fc881557a09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.663013 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9220288c84b847fea7df2fc881557a09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.666461 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:52.667429 12394 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_d6b47ef12b28492bb711ca846f80f6cb [id=c2c079dc87ec480a82aa5b57b110da17] alter complete (version 3)
I20260926 19:58:52.667439 13596 consensus_queue.cc:1048] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:52.668654 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77b319b685344a848462b78617eb6c22" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:52.669754 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:52.674652 12759 raft_consensus.cc:2468] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:52.675704 13569 ts_tablet_manager.cc:1929] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:52.677462 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:52.679738 12519 leader_election.cc:304] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:52.680720 12519 leader_election.cc:304] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:52.681666 13662 raft_consensus.cc:2749] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:52.681802 13660 raft_consensus.cc:2804] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:52.682425 13660 raft_consensus.cc:697] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:52.683143 13660 consensus_queue.cc:237] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.685087 13203 ts_tablet_manager.cc:1929] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:52.690125 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 5450b4cd35f04904845f1612fe58be52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:52.693881 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 (DEFAULT_TABLE table=default.Table_b70cb5834613437fab92972194f769da [id=f8b39f76230f41ad8b6cf0670b751df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.694808 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51f5d8149d474ab1b52d53fc0b490032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.692051 12734 tablet_service.cc:1511] Processing CreateTablet for tablet b58bb831147e44d496fea1f83fd0504c (DEFAULT_TABLE table=default.Table_8b39d71ccfc048848d34cf14d1f2e1fd [id=96e6a79e95024055a1cfa851677bd080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.700780 13569 log.cc:1199] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/56e337767be84507acdecfb189705bdd
I20260926 19:58:52.701380 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58bb831147e44d496fea1f83fd0504c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.701571 13569 ts_tablet_manager.cc:1950] T 56e337767be84507acdecfb189705bdd P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:52.701758 13403 ts_tablet_manager.cc:1929] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:52.702091 13403 log.cc:1199] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/56e337767be84507acdecfb189705bdd
I20260926 19:58:52.702744 13403 ts_tablet_manager.cc:1950] T 56e337767be84507acdecfb189705bdd P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:52.704459 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 208) took 1144 ms. Trace:
I20260926 19:58:52.704748 13403 rpcz_store.cc:276] 0926 19:58:51.559621 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.560256 (+ 635us) service_pool.cc:224] Handling call
0926 19:58:52.478875 (+918619us) ts_tablet_manager.cc:1151] Deleting tablet 56e337767be84507acdecfb189705bdd
0926 19:58:52.478913 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:52.561456 (+ 82543us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.701598 (+140142us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.704404 (+ 2806us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":918013}
I20260926 19:58:52.705341 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 209) took 1647 ms. Trace:
I20260926 19:58:52.707090 13092 tablet_bootstrap.cc:492] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:52.707491 13092 ts_tablet_manager.cc:1403] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.093s user 0.013s sys 0.002s
I20260926 19:58:52.707752 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 56e337767be84507acdecfb189705bdd (table default.Table_6cabacdbfcc94b7e876d1a62734db5a6 [id=0d5fc67f634c45dab68955448173bfd7]) successfully deleted
I20260926 19:58:52.708190 13013 tablet_bootstrap.cc:492] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:52.696862 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:52.699599 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 (DEFAULT_TABLE table=default.Table_b70cb5834613437fab92972194f769da [id=f8b39f76230f41ad8b6cf0670b751df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.708578 13013 ts_tablet_manager.cc:1403] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.154s user 0.009s sys 0.000s
I20260926 19:58:52.709263 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51f5d8149d474ab1b52d53fc0b490032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.709831 13092 raft_consensus.cc:359] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.710390 13092 raft_consensus.cc:385] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.710603 13092 raft_consensus.cc:740] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.711138 13092 consensus_queue.cc:260] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.722616 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 761863172e9342aab759dd2b4dcb0404 (DEFAULT_TABLE table=default.Table_09d6cf0518f640eaa9d4cf503b55422b [id=db2cdeaf1744424d899fe5cd9e247676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.722760 12740 tablet_service.cc:1511] Processing CreateTablet for tablet f5b4b81f11254306b56c234f22b86ea5 (DEFAULT_TABLE table=default.Table_f07ac4e733db421bb4a3efc6f937c310 [id=c2dfbc1199344fc187669bf813fcc16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.721690 12896 tablet_service.cc:1511] Processing CreateTablet for tablet 761863172e9342aab759dd2b4dcb0404 (DEFAULT_TABLE table=default.Table_09d6cf0518f640eaa9d4cf503b55422b [id=db2cdeaf1744424d899fe5cd9e247676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.723645 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b4b81f11254306b56c234f22b86ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.724041 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761863172e9342aab759dd2b4dcb0404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.725633 13092 ts_tablet_manager.cc:1434] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20260926 19:58:52.726199 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761863172e9342aab759dd2b4dcb0404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.732649 13092 tablet_bootstrap.cc:492] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:52.734229 12735 tablet_service.cc:1558] Processing DeleteTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:51 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:52.705724 13569 rpcz_store.cc:276] 0926 19:58:51.058005 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.107558 (+ 49553us) service_pool.cc:224] Handling call
0926 19:58:51.718350 (+610792us) ts_tablet_manager.cc:1151] Deleting tablet 56e337767be84507acdecfb189705bdd
0926 19:58:51.718400 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:52.074038 (+355638us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b2400001b98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95acc4b4 00007f2c95aca5bb 00007f2c95c8abf9 00007f2c95c8a885 00007f2c95c89de0 00007f2c9cb65d95 00007f2c9cb7f7c9 00007f2c9cb83f63 00007f2c9d1f56b5 00007f2c9de5ec3a
0926 19:58:52.285300 (+211262us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.675575 (+390275us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.705276 (+ 29701us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80623,"spinlock_wait_cycles":25846144,"tablet-delete.queue_time_us":610166}
I20260926 19:58:52.735236 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5403a485fc4fe69a8d042dc919a907 (DEFAULT_TABLE table=default.Table_f8331edd1b074ce1b2b43aaf0100626b [id=cd96af7e357e4cc2ab70be9e3d105ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.736089 13569 tablet_replica.cc:333] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:52.736549 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5403a485fc4fe69a8d042dc919a907 (DEFAULT_TABLE table=default.Table_f8331edd1b074ce1b2b43aaf0100626b [id=cd96af7e357e4cc2ab70be9e3d105ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.737131 13569 raft_consensus.cc:2243] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:52.737257 12895 tablet_service.cc:1511] Processing CreateTablet for tablet f5b4b81f11254306b56c234f22b86ea5 (DEFAULT_TABLE table=default.Table_f07ac4e733db421bb4a3efc6f937c310 [id=c2dfbc1199344fc187669bf813fcc16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.737699 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5403a485fc4fe69a8d042dc919a907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.737661 13569 raft_consensus.cc:2272] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:52.738245 12895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b4b81f11254306b56c234f22b86ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.738946 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 56e337767be84507acdecfb189705bdd (table default.Table_6cabacdbfcc94b7e876d1a62734db5a6 [id=0d5fc67f634c45dab68955448173bfd7]) successfully deleted
I20260926 19:58:52.700392 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet 5450b4cd35f04904845f1612fe58be52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:52.745570 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5403a485fc4fe69a8d042dc919a907. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:52.748330 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 3066a65ce44b4387a6f96124d9f476fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3066a65ce44b4387a6f96124d9f476fb already in progress: creating tablet
I20260926 19:58:52.748564 13012 tablet_bootstrap.cc:654] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.748526 13013 raft_consensus.cc:359] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.749123 13013 raft_consensus.cc:385] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.749393 13013 raft_consensus.cc:740] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:52.750211 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 6e20b1a22af944a0aad0fee5fc835f6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e20b1a22af944a0aad0fee5fc835f6e already in progress: creating tablet
W20260926 19:58:52.752447 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 56fd4c88f9304f19883c3e5da8f937b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56fd4c88f9304f19883c3e5da8f937b5 already in progress: creating tablet
I20260926 19:58:52.752696 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 8ae465ba0677434590c0732a654042b5 (DEFAULT_TABLE table=default.Table_43587029a971483b88871b746dc9088c [id=2b717ac3031c43be804f98eb5da0a723]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:52.753006 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 51f5d8149d474ab1b52d53fc0b490032 because the tablet was not found. No further retry: Not found: Tablet not found: 51f5d8149d474ab1b52d53fc0b490032
I20260926 19:58:52.753630 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae465ba0677434590c0732a654042b5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:52.753589 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 51f5d8149d474ab1b52d53fc0b490032 because the tablet was not found. No further retry: Not found: Tablet not found: 51f5d8149d474ab1b52d53fc0b490032
I20260926 19:58:52.749351 13092 tablet_bootstrap.cc:654] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:52.754169 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 5450b4cd35f04904845f1612fe58be52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5450b4cd35f04904845f1612fe58be52 already in progress: creating tablet
I20260926 19:58:52.768448 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 7464091f0d1941788e920259fbd6e6d2 (DEFAULT_TABLE table=default.Table_063a6e0131614ece86586e6991c8efe8 [id=3231ff9e929c47b580188d31a80fbbc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.769464 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7464091f0d1941788e920259fbd6e6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.770489 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 8ae465ba0677434590c0732a654042b5 (DEFAULT_TABLE table=default.Table_43587029a971483b88871b746dc9088c [id=2b717ac3031c43be804f98eb5da0a723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.771399 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae465ba0677434590c0732a654042b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.777428 12735 tablet_service.cc:1511] Processing CreateTablet for tablet 7464091f0d1941788e920259fbd6e6d2 (DEFAULT_TABLE table=default.Table_063a6e0131614ece86586e6991c8efe8 [id=3231ff9e929c47b580188d31a80fbbc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.778201 13569 ts_tablet_manager.cc:1916] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:52.778287 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_652bcc921b9c4e77943bc28b1c6e2166"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.779783 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_652bcc921b9c4e77943bc28b1c6e2166 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.782214 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_65a24dd219044e0f80e841c1254730d6" } new_table_name: "default.Table_cc7b2242e1c44b9cabad45093975a899" modify_external_catalogs: true
I20260926 19:58:52.782876 13629 leader_election.cc:290] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:52.784013 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff071a16f1244cebc8c93e5508280a3" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:52.784916 12759 raft_consensus.cc:2393] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:52.786181 12857 leader_election.cc:304] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:52.791237 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7464091f0d1941788e920259fbd6e6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.793473 13013 consensus_queue.cc:260] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.795513 13013 ts_tablet_manager.cc:1434] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.087s user 0.005s sys 0.000s
I20260926 19:58:52.798275 13633 raft_consensus.cc:3060] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:52.807104 13633 raft_consensus.cc:2749] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:52.810740 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 249f255b082e497c9ab57fe62564ec47 (DEFAULT_TABLE table=default.Table_24fce316a7434e408e47ec249b8a824f [id=5bf014dc4431426a865ba15a01fce4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.810923 13013 tablet_bootstrap.cc:492] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:52.821702 12394 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_1dfed69d00a647d1814ee455d0d3b5e2 [id=3f2e4633bd6c4004ab931d817248ed45] alter complete (version 1)
W20260926 19:58:52.822345 12394 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 59) took 1253 ms (client timeout 1000 ms). Trace:
I20260926 19:58:52.835074 12560 tablet_service.cc:1511] Processing CreateTablet for tablet b58bb831147e44d496fea1f83fd0504c (DEFAULT_TABLE table=default.Table_8b39d71ccfc048848d34cf14d1f2e1fd [id=96e6a79e95024055a1cfa851677bd080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.836090 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58bb831147e44d496fea1f83fd0504c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:52.839089 12391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 50) took 1249 ms (client timeout 1000 ms). Trace:
W20260926 19:58:52.822638 12394 rpcz_store.cc:269] 0926 19:58:51.568931 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.569041 (+ 110us) service_pool.cc:224] Handling call
0926 19:58:51.574011 (+ 4970us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:52.822282 (+1248271us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:51.619849 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:51.620195 (+ 346us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:51.620216 (+ 21us) write_op.cc:435] Acquired schema lock
0926 19:58:51.620228 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:51.620458 (+ 230us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:51.620489 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:51.620508 (+ 19us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:51.620629 (+ 121us) tablet.cc:701] Row locks acquired
0926 19:58:51.620641 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:51.620731 (+ 90us) write_op.cc:270] Start()
0926 19:58:51.620816 (+ 85us) write_op.cc:276] Timestamp: P: 1790452731620695 usec, L: 0
0926 19:58:51.620830 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:51.621208 (+ 378us) log.cc:844] Serialized 356 byte log entry
0926 19:58:51.638129 (+ 16921us) op_driver.cc:464] REPLICATION: finished
0926 19:58:51.639798 (+ 1669us) write_op.cc:301] APPLY: starting
0926 19:58:51.639841 (+ 43us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:51.639922 (+ 81us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:51.639996 (+ 74us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:51.640120 (+ 124us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:51.640232 (+ 112us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:51.640270 (+ 38us) write_op.cc:312] APPLY: finished
0926 19:58:51.640407 (+ 137us) log.cc:844] Serialized 23 byte log entry
0926 19:58:51.640514 (+ 107us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.640559 (+ 45us) write_op.cc:454] Released schema lock
0926 19:58:51.640590 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:51.640782 (+ 192us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:51.640805 (+ 23us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 19:58:52.786358 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:52.786702 (+ 344us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:52.786725 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:58:52.786738 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:52.786974 (+ 236us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:52.787008 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:52.787029 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:52.787169 (+ 140us) tablet.cc:701] Row locks acquired
0926 19:58:52.787182 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:52.787288 (+ 106us) write_op.cc:270] Start()
0926 19:58:52.787364 (+ 76us) write_op.cc:276] Timestamp: P: 1790452732787243 usec, L: 0
0926 19:58:52.787378 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:52.787813 (+ 435us) log.cc:844] Serialized 356 byte log entry
0926 19:58:52.810375 (+ 22562us) op_driver.cc:464] REPLICATION: finished
0926 19:58:52.820429 (+ 10054us) write_op.cc:301] APPLY: starting
0926 19:58:52.820480 (+ 51us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:52.820587 (+ 107us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:52.820688 (+ 101us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:52.820835 (+ 147us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:52.820970 (+ 135us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:52.821014 (+ 44us) write_op.cc:312] APPLY: finished
0926 19:58:52.821174 (+ 160us) log.cc:844] Serialized 23 byte log entry
0926 19:58:52.821314 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:52.821371 (+ 57us) write_op.cc:454] Released schema lock
0926 19:58:52.821399 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:52.821630 (+ 231us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:52.821656 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":6058752,"child_traces":[["op",{"apply.queue_time_us":1536,"apply.run_cpu_time_us":948,"apply.run_wall_time_us":949,"num_ops":1,"prepare.queue_time_us":7303,"prepare.run_cpu_time_us":1584,"prepare.run_wall_time_us":1584,"raft.queue_time_us":990,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":10,"replication_time_us":17225}],["op",{"apply.queue_time_us":9884,"apply.run_cpu_time_us":1183,"apply.run_wall_time_us":1183,"num_ops":1,"prepare.queue_time_us":116155,"prepare.run_cpu_time_us":1633,"prepare.run_wall_time_us":1632,"replication_time_us":22895}]]}
I20260926 19:58:52.844950 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd184e6295d47aeba809be6aab9ce1c with delete_type TABLET_DATA_DELETED (replaced by tablet 22f268b4a1f7423f917731c6b62aab3e) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:52.846091 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 16df0828847d4e7db5702d84082fa402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:52.847678 12393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 54) took 1189 ms (client timeout 1000 ms). Trace:
W20260926 19:58:52.848029 12393 rpcz_store.cc:269] 0926 19:58:51.657878 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.703861 (+ 45983us) service_pool.cc:224] Handling call
0926 19:58:52.847608 (+1143747us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6258,"reported_tablets":21}
W20260926 19:58:52.839430 12391 rpcz_store.cc:269] 0926 19:58:51.589603 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.589785 (+ 182us) service_pool.cc:224] Handling call
0926 19:58:52.839025 (+1249240us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:52.304358 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:52.304852 (+ 494us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:52.304884 (+ 32us) write_op.cc:435] Acquired schema lock
0926 19:58:52.304903 (+ 19us) tablet.cc:662] Decoding operations
0926 19:58:52.305283 (+ 380us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:52.305336 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:52.306261 (+ 925us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:52.306606 (+ 345us) tablet.cc:701] Row locks acquired
0926 19:58:52.306625 (+ 19us) write_op.cc:260] PREPARE: finished
0926 19:58:52.306756 (+ 131us) write_op.cc:270] Start()
0926 19:58:52.306857 (+ 101us) write_op.cc:276] Timestamp: P: 1790452732306691 usec, L: 0
0926 19:58:52.306878 (+ 21us) op_driver.cc:348] REPLICATION: starting
0926 19:58:52.307442 (+ 564us) log.cc:844] Serialized 919 byte log entry
0926 19:58:52.335127 (+ 27685us) op_driver.cc:464] REPLICATION: finished
0926 19:58:52.338134 (+ 3007us) write_op.cc:301] APPLY: starting
0926 19:58:52.338208 (+ 74us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:52.338350 (+ 142us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:52.338466 (+ 116us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:52.338707 (+ 241us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:52.338870 (+ 163us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:58:52.338920 (+ 50us) write_op.cc:312] APPLY: finished
0926 19:58:52.339102 (+ 182us) log.cc:844] Serialized 29 byte log entry
0926 19:58:52.339686 (+ 584us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:52.340205 (+ 519us) write_op.cc:454] Released schema lock
0926 19:58:52.340237 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:52.369777 (+ 29540us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:52.369816 (+ 39us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":2559,"apply.run_cpu_time_us":1931,"apply.run_wall_time_us":31569,"num_ops":2,"prepare.queue_time_us":96152,"prepare.run_cpu_time_us":2606,"prepare.run_wall_time_us":37014,"raft.queue_time_us":340,"raft.run_cpu_time_us":721,"raft.run_wall_time_us":1521,"replication_time_us":28128}]]}
I20260926 19:58:52.859678 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 51f5d8149d474ab1b52d53fc0b490032 (DEFAULT_TABLE table=default.Table_b70cb5834613437fab92972194f769da [id=f8b39f76230f41ad8b6cf0670b751df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.860364 12561 tablet_service.cc:1511] Processing CreateTablet for tablet f5b4b81f11254306b56c234f22b86ea5 (DEFAULT_TABLE table=default.Table_f07ac4e733db421bb4a3efc6f937c310 [id=c2dfbc1199344fc187669bf813fcc16b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.860675 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51f5d8149d474ab1b52d53fc0b490032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.860770 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_bfdeb281a21842bc9cf53aafa87f80cd [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 3 got 2
I20260926 19:58:52.860726 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 8ae465ba0677434590c0732a654042b5 (DEFAULT_TABLE table=default.Table_43587029a971483b88871b746dc9088c [id=2b717ac3031c43be804f98eb5da0a723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.861282 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b4b81f11254306b56c234f22b86ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.861811 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae465ba0677434590c0732a654042b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.862783 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 761863172e9342aab759dd2b4dcb0404 (DEFAULT_TABLE table=default.Table_09d6cf0518f640eaa9d4cf503b55422b [id=db2cdeaf1744424d899fe5cd9e247676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.863638 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 761863172e9342aab759dd2b4dcb0404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.862051 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5403a485fc4fe69a8d042dc919a907 (DEFAULT_TABLE table=default.Table_f8331edd1b074ce1b2b43aaf0100626b [id=cd96af7e357e4cc2ab70be9e3d105ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:52.866560 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5403a485fc4fe69a8d042dc919a907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.868094 12393 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_2b4f759d68744ea8b7d986616f8618d1" } new_table_name: "default.Table_3246f278dd5e48c9bf8e940cc4ee0f7c" modify_external_catalogs: true
W20260926 19:58:52.870325 12380 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 59d1955b12dd47899b8a226fe0e980ae consensus state has no leader
I20260926 19:58:52.871484 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a195c0af637546dc947815ce7a384b8b" } modify_external_catalogs: true
I20260926 19:58:52.872584 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 8ff071a16f1244cebc8c93e5508280a3 (table default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7 [id=e452740c55c345a9ab7445009c7a71cb]). Expected version 2 got 0
I20260926 19:58:52.873116 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:52.874285 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f05b7ded126448f2acba6af9bc981450" } modify_external_catalogs: true
I20260926 19:58:52.874531 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:52.874760 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f05b7ded126448f2acba6af9bc981450" } modify_external_catalogs: true
I20260926 19:58:52.876754 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 249f255b082e497c9ab57fe62564ec47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:52.871953 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a195c0af637546dc947815ce7a384b8b" } modify_external_catalogs: true
I20260926 19:58:52.887645 13092 tablet_bootstrap.cc:492] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:52.888454 13092 ts_tablet_manager.cc:1403] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.156s user 0.007s sys 0.002s
I20260926 19:58:52.891152 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_e572a5b025f943e38097a94a29d245ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:52.892154 13013 tablet_bootstrap.cc:654] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:52.892896 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e572a5b025f943e38097a94a29d245ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.895985 12545 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37786 (request call id 34) took 1046 ms. Trace:
I20260926 19:58:52.896370 12545 rpcz_store.cc:276] 0926 19:58:51.849632 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.849759 (+ 127us) service_pool.cc:224] Handling call
0926 19:58:52.589218 (+739459us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b5400111554. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a258406b8 00007f9a268b198d 00007f9a25001104 00007f9a25001092 00007f9a25000fc1 00007f9a25000f3c 00007f9a250001f2 00007f9a2351509c 00007f9a235144d6 00007f9a23513e7f
0926 19:58:52.895782 (+306564us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":996487936}
I20260926 19:58:52.902969 13013 tablet_bootstrap.cc:492] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:52.903343 13013 ts_tablet_manager.cc:1403] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.107s user 0.011s sys 0.003s
I20260926 19:58:52.905923 13013 raft_consensus.cc:359] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.906514 13013 raft_consensus.cc:385] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.906760 13013 raft_consensus.cc:740] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.907310 13013 consensus_queue.cc:260] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.909380 13013 ts_tablet_manager.cc:1434] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260926 19:58:52.910198 13013 tablet_bootstrap.cc:492] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:52.914991 13092 raft_consensus.cc:359] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.915599 13092 raft_consensus.cc:385] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.915853 13092 raft_consensus.cc:740] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.916082 13013 tablet_bootstrap.cc:654] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.916460 13092 consensus_queue.cc:260] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.935884 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_beef77b0962d44d8948a5f4984c0956f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:52.938141 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_beef77b0962d44d8948a5f4984c0956f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.954459 13092 ts_tablet_manager.cc:1434] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.066s user 0.006s sys 0.001s
I20260926 19:58:52.960821 13013 tablet_bootstrap.cc:492] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:52.961213 13013 ts_tablet_manager.cc:1403] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.003s
I20260926 19:58:52.963385 13013 raft_consensus.cc:359] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.964001 13013 raft_consensus.cc:385] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.964247 13013 raft_consensus.cc:740] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.964843 13013 consensus_queue.cc:260] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.968246 13013 ts_tablet_manager.cc:1434] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260926 19:58:52.969153 13013 tablet_bootstrap.cc:492] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:52.969298 13092 tablet_bootstrap.cc:492] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:52.685405 13203 log.cc:1199] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/886ab0feae1e49629290bc892e518816
I20260926 19:58:52.970592 13203 ts_tablet_manager.cc:1950] T 886ab0feae1e49629290bc892e518816 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:52.971418 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 212) took 2083 ms. Trace:
W20260926 19:58:52.972335 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 329695us
I20260926 19:58:52.974556 13013 tablet_bootstrap.cc:654] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:52.975788 13012 tablet_bootstrap.cc:492] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:52.976190 13012 ts_tablet_manager.cc:1403] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.475s user 0.004s sys 0.005s
I20260926 19:58:52.982758 12389 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_582e94c0978641ddafbbf586413fffb5 [id=03889de8878549b7814b02a32caf20c3] alter complete (version 2)
I20260926 19:58:52.990137 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e337767be84507acdecfb189705bdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:52.990433 13013 tablet_bootstrap.cc:492] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:52.990979 13013 ts_tablet_manager.cc:1403] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.008s
I20260926 19:58:52.991601 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 77b319b685344a848462b78617eb6c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:52.996735 13013 raft_consensus.cc:359] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:52.997397 13013 raft_consensus.cc:385] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:52.997731 13013 raft_consensus.cc:740] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:52.998792 13013 consensus_queue.cc:260] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.001322 13013 ts_tablet_manager.cc:1434] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260926 19:58:53.002228 13013 tablet_bootstrap.cc:492] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:53.004895 13092 tablet_bootstrap.cc:654] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.008221 13013 tablet_bootstrap.cc:654] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.009605 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 249f255b082e497c9ab57fe62564ec47 (DEFAULT_TABLE table=default.Table_24fce316a7434e408e47ec249b8a824f [id=5bf014dc4431426a865ba15a01fce4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.010664 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 249f255b082e497c9ab57fe62564ec47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.020605 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 5930402db43740b4b291c6c238ecf52c
I20260926 19:58:53.025318 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 5930402db43740b4b291c6c238ecf52c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:53.028029 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 62e75eb4b3e8493b9d599c1c4ad4c67f (table default.Table_bfdeb281a21842bc9cf53aafa87f80cd [id=dd5c0a6b728948278a5dd48f8ec0bf6f]). Expected version 3 got 2
I20260926 19:58:53.037238 13013 tablet_bootstrap.cc:492] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:53.037658 13013 ts_tablet_manager.cc:1403] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.001s
I20260926 19:58:53.041189 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_86e17c4523fa4812a508156cff380a35"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:53.042713 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86e17c4523fa4812a508156cff380a35 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:52.971794 13203 rpcz_store.cc:276] 0926 19:58:50.887715 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.887809 (+ 94us) service_pool.cc:224] Handling call
0926 19:58:52.194398 (+1306589us) ts_tablet_manager.cc:1151] Deleting tablet 886ab0feae1e49629290bc892e518816
0926 19:58:52.194450 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:52.493146 (+298696us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.684973 (+191827us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:52.971352 (+286379us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1257738}
I20260926 19:58:53.046653 12396 master_service.cc:960] ReplaceTablet: received request to replace tablet c5b734f0d33342d3be8b1827bcd5dd14 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:53.048929 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 8c7d0435f3564cb38b6cbca713009c2e
I20260926 19:58:53.050684 13012 raft_consensus.cc:359] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.051219 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:53.051347 13012 raft_consensus.cc:385] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:53.051308 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 53668c46bd3441ad936bf990b3192420 (table default.Table_3246f278dd5e48c9bf8e940cc4ee0f7c [id=faf55c4cd53c42a9b894d47f5064d220]). Expected version 1 got 0
I20260926 19:58:53.051573 13012 raft_consensus.cc:740] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:53.052331 13012 consensus_queue.cc:260] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.052742 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 8ff071a16f1244cebc8c93e5508280a3 (table default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7 [id=e452740c55c345a9ab7445009c7a71cb]). Expected version 2 got 0
I20260926 19:58:53.053320 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:53.054138 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:53.054320 13203 tablet_replica.cc:333] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:53.050442 13013 raft_consensus.cc:359] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.056247 13013 raft_consensus.cc:385] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:53.056495 13013 raft_consensus.cc:740] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:53.056627 13203 raft_consensus.cc:2243] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:53.057075 13013 consensus_queue.cc:260] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.057566 13203 raft_consensus.cc:2272] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:53.062590 13013 ts_tablet_manager.cc:1434] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.025s user 0.005s sys 0.001s
I20260926 19:58:53.063490 13013 tablet_bootstrap.cc:492] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:53.068586 13013 tablet_bootstrap.cc:654] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.054878 12393 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 55b7715b8fcb4b59917ae14a83f9955d (table default.Table_cc7b2242e1c44b9cabad45093975a899 [id=d9c3ce1d8dd440c2862eb5fa4c58bf72]). Expected version 1 got 0
I20260926 19:58:53.076303 13203 ts_tablet_manager.cc:1916] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:53.080228 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_0911867d73d545f7bf86c97f052dee59" } new_table_name: "default.Table_d19c33847de64e7c9899f408efa9607d" modify_external_catalogs: true
I20260926 19:58:53.098191 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 249f255b082e497c9ab57fe62564ec47 (DEFAULT_TABLE table=default.Table_24fce316a7434e408e47ec249b8a824f [id=5bf014dc4431426a865ba15a01fce4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.098694 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 3215bbd5be56438491a73925ac9ccbe6 with delete_type TABLET_DATA_DELETED (replaced by tablet 1d0583424ef64d9ea3beeb1c01ceafba) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.107388 13013 tablet_bootstrap.cc:492] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:53.107781 13013 ts_tablet_manager.cc:1403] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.005s
I20260926 19:58:53.107643 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 7464091f0d1941788e920259fbd6e6d2 (DEFAULT_TABLE table=default.Table_063a6e0131614ece86586e6991c8efe8 [id=3231ff9e929c47b580188d31a80fbbc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.110203 13013 raft_consensus.cc:359] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.110838 13013 raft_consensus.cc:385] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:53.111078 13013 raft_consensus.cc:740] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:53.111644 13013 consensus_queue.cc:260] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.114114 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 77b319b685344a848462b78617eb6c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.117215 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 886ab0feae1e49629290bc892e518816 (table default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189]) successfully deleted
I20260926 19:58:53.103645 13203 ts_tablet_manager.cc:1929] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:53.118686 13203 log.cc:1199] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/56e337767be84507acdecfb189705bdd
I20260926 19:58:53.119691 13203 ts_tablet_manager.cc:1950] T 56e337767be84507acdecfb189705bdd P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:53.120455 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 213) took 2232 ms. Trace:
I20260926 19:58:53.120807 13203 rpcz_store.cc:276] 0926 19:58:50.888431 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:50.888600 (+ 169us) service_pool.cc:224] Handling call
0926 19:58:53.054113 (+2165513us) ts_tablet_manager.cc:1151] Deleting tablet 56e337767be84507acdecfb189705bdd
0926 19:58:53.054174 (+ 61us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.096221 (+ 42047us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.103542 (+ 7321us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.120405 (+ 16863us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16092,"tablet-delete.queue_time_us":2117388}
I20260926 19:58:53.122014 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.122031 13203 tablet_replica.cc:333] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:53.129778 13013 ts_tablet_manager.cc:1434] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.022s user 0.007s sys 0.001s
I20260926 19:58:53.130683 13203 raft_consensus.cc:2243] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:53.131515 13203 raft_consensus.cc:2272] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:53.133809 13549 raft_consensus.cc:493] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.134232 13549 raft_consensus.cc:515] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.106287 13704 tablet.cc:1751] T 53668c46bd3441ad936bf990b3192420 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:53.136204 13549 leader_election.cc:290] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.099447 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 249f255b082e497c9ab57fe62564ec47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.137392 13549 raft_consensus.cc:493] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.137856 13549 raft_consensus.cc:515] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.139185 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7464091f0d1941788e920259fbd6e6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.143952 13549 leader_election.cc:290] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.118018 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 5930402db43740b4b291c6c238ecf52c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.146946 13013 tablet_bootstrap.cc:492] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:53.148855 12395 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_979fb16236da4673983d947c83506fd3 [id=02441df5c4e6405fa8a72aa72b16687e] alter complete (version 2)
I20260926 19:58:53.159965 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:53.161664 12519 leader_election.cc:343] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:53.166581 12393 master_service.cc:960] ReplaceTablet: received request to replace tablet 13bb4e7eeef145fc85a5c412add99ce7 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:53.168287 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16897e7489e9445ba5389a4f3507a739" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.169003 12759 raft_consensus.cc:2468] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:53.176757 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 56e337767be84507acdecfb189705bdd (table default.Table_6cabacdbfcc94b7e876d1a62734db5a6 [id=0d5fc67f634c45dab68955448173bfd7]) successfully deleted
I20260926 19:58:53.176846 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 66c8916d88dd48998eab514bd3c538b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 700d05149a934d5e8bec9cc41873ad3e) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.177750 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.177995 12519 leader_election.cc:304] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:53.180823 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 886ab0feae1e49629290bc892e518816 with delete_type TABLET_DATA_DELETED (replaced by tablet 2ad76fc959e24545bdfad19f988fce93) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.181270 13549 raft_consensus.cc:2804] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:53.182163 13549 raft_consensus.cc:493] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:53.182909 13549 raft_consensus.cc:3060] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:53.196568 13013 tablet_bootstrap.cc:654] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.201699 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 16897e7489e9445ba5389a4f3507a739 with delete_type TABLET_DATA_DELETED (replaced by tablet 986a28107bd54199bf2cda96978150c3) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.184125 13203 ts_tablet_manager.cc:1916] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:53.208541 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a29b5ffc06a5457db50ba2559768ba53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:53.211213 13012 ts_tablet_manager.cc:1434] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.235s user 0.007s sys 0.000s
I20260926 19:58:53.211367 12394 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_447c2fd2552a48d0897da03fd0d1d963" } new_table_name: "default.Table_17815f4814b9429da94beee1f7a60322" modify_external_catalogs: true
I20260926 19:58:53.218308 13629 raft_consensus.cc:493] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.219444 13629 raft_consensus.cc:515] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.220067 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16897e7489e9445ba5389a4f3507a739" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:53.220302 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:53.221231 12921 raft_consensus.cc:2468] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:53.223877 13629 leader_election.cc:290] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.231093 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet 16897e7489e9445ba5389a4f3507a739 with delete_type TABLET_DATA_DELETED (replaced by tablet 986a28107bd54199bf2cda96978150c3) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:53.231452 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_91489cb29847444d9e79a0587e38a517" } modify_external_catalogs: true
I20260926 19:58:53.232053 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_91489cb29847444d9e79a0587e38a517" } modify_external_catalogs: true
W20260926 19:58:53.233150 12521 leader_election.cc:343] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:53.239411 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:53.240604 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:53.241253 12589 raft_consensus.cc:2468] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:53.243700 13092 tablet_bootstrap.cc:492] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:53.245919 13092 ts_tablet_manager.cc:1403] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.277s user 0.008s sys 0.004s
I20260926 19:58:53.251992 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:53.254904 12857 leader_election.cc:343] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:53.255254 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 77b319b685344a848462b78617eb6c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:43 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:53.256764 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_693c5b4b4dc446378398965b02ddeea5" } modify_external_catalogs: true
I20260926 19:58:53.257332 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_693c5b4b4dc446378398965b02ddeea5" } modify_external_catalogs: true
I20260926 19:58:53.261456 13705 tablet.cc:1751] T 53668c46bd3441ad936bf990b3192420 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:53.265467 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 66c8916d88dd48998eab514bd3c538b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 700d05149a934d5e8bec9cc41873ad3e) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:53.266950 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 16df0828847d4e7db5702d84082fa402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:53.267230 12900 tablet_service.cc:1558] Processing DeleteTablet for tablet 56e337767be84507acdecfb189705bdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:41 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:53.270674 13633 consensus_queue.cc:1048] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:53.274295 12858 leader_election.cc:304] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
W20260926 19:58:53.275707 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 8c7d0435f3564cb38b6cbca713009c2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c7d0435f3564cb38b6cbca713009c2e already in progress: creating tablet
I20260926 19:58:53.275827 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet 5930402db43740b4b291c6c238ecf52c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:53.278836 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_d4eea629040e41128693ff3cab8caa65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:53.280791 13633 raft_consensus.cc:2804] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:53.281263 13633 raft_consensus.cc:493] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:53.281713 13633 raft_consensus.cc:3060] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:53.281823 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 48722us
I20260926 19:58:53.282568 12521 leader_election.cc:304] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:53.283304 13660 raft_consensus.cc:2804] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:53.283665 13660 raft_consensus.cc:493] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:53.284014 13660 raft_consensus.cc:3060] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:53.288532 13403 tablet_replica.cc:333] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:53.293974 13660 raft_consensus.cc:515] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:53.297611 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8a0e37779b0e415a930c8d3b5317bcd0 consensus state has no leader
W20260926 19:58:53.300033 12394 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d4434cb08d0f41fd96283de49c53b3e8 consensus state has no leader
I20260926 19:58:53.309971 13012 tablet_bootstrap.cc:492] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:53.313594 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4eea629040e41128693ff3cab8caa65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:53.315263 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a29b5ffc06a5457db50ba2559768ba53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.322774 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a73209f0f4544df897c1e856f3692fe8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:53.324812 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a73209f0f4544df897c1e856f3692fe8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.303743 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:53.288260 13092 raft_consensus.cc:359] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.341986 13092 raft_consensus.cc:385] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:53.342419 13092 raft_consensus.cc:740] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:53.343014 13092 consensus_queue.cc:260] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.344548 13092 ts_tablet_manager.cc:1434] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.098s user 0.005s sys 0.000s
W20260926 19:58:53.350489 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 112746us
I20260926 19:58:53.357049 13716 raft_consensus.cc:493] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.357539 13716 raft_consensus.cc:515] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.362216 13723 raft_consensus.cc:493] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.362692 13723 raft_consensus.cc:515] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.370042 13723 leader_election.cc:290] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.370476 13723 raft_consensus.cc:493] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.370865 13723 raft_consensus.cc:515] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.370880 13549 raft_consensus.cc:515] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.374081 13660 leader_election.cc:290] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.380147 13722 tablet.cc:1751] T 55b7715b8fcb4b59917ae14a83f9955d P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:53.382053 13549 leader_election.cc:290] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.386811 13723 leader_election.cc:290] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.387588 13569 ts_tablet_manager.cc:1929] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:53.387548 13721 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:53.388054 13569 log.cc:1199] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/cd92a1a62ab24647bef7c9eefdf9c9d8
I20260926 19:58:53.389344 13569 ts_tablet_manager.cc:1950] T cd92a1a62ab24647bef7c9eefdf9c9d8 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:53.391561 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 206) took 2346 ms. Trace:
I20260926 19:58:53.392431 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet fb593ce2cd4147febebfe442de9fdd28
I20260926 19:58:53.395040 13013 tablet_bootstrap.cc:492] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:53.419144 13013 ts_tablet_manager.cc:1403] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.272s user 0.022s sys 0.006s
I20260926 19:58:53.421485 13013 raft_consensus.cc:359] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.422133 13013 raft_consensus.cc:385] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:53.422381 13013 raft_consensus.cc:740] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:53.423187 13013 consensus_queue.cc:260] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.425263 12564 tablet_service.cc:1558] Processing DeleteTablet for tablet fb593ce2cd4147febebfe442de9fdd28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:53.399524 13716 leader_election.cc:290] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.431988 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b359ca01127641a4b703f455ec9bee7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:53.433929 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b359ca01127641a4b703f455ec9bee7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.455467 13720 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:53.456776 13403 raft_consensus.cc:2243] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:53.396885 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet fb593ce2cd4147febebfe442de9fdd28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:53.398453 13679 consensus_queue.cc:1048] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:53.405814 13012 tablet_bootstrap.cc:654] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.417256 13715 raft_consensus.cc:493] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8112c0b43d9c4e5389aa3a18f18b58a5)
I20260926 19:58:53.461803 13715 raft_consensus.cc:515] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.462811 13092 tablet_bootstrap.cc:492] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:53.463934 13715 leader_election.cc:290] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:58:53.478459 12695 leader_election.cc:343] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 1455b99a65ff44ff933825196708f485
I20260926 19:58:53.481074 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bdb479c5384d0891db66943ef263b2" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.482155 13569 rpcz_store.cc:276] 0926 19:58:51.045245 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.108397 (+ 63152us) service_pool.cc:224] Handling call
0926 19:58:52.735923 (+1627526us) ts_tablet_manager.cc:1151] Deleting tablet cd92a1a62ab24647bef7c9eefdf9c9d8
0926 19:58:52.735966 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.102187 (+366221us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.387411 (+285224us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.391494 (+ 4083us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1626904}
I20260926 19:58:53.484395 13704 tablet.cc:1751] T 55b7715b8fcb4b59917ae14a83f9955d P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:53.489806 13013 ts_tablet_manager.cc:1434] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.070s user 0.009s sys 0.007s
I20260926 19:58:53.493266 13013 tablet_bootstrap.cc:492] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:53.499233 13724 raft_consensus.cc:493] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.496142 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16897e7489e9445ba5389a4f3507a739" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:53.497452 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd86f01d82d041f4852f83f2547e697f" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.503398 13013 tablet_bootstrap.cc:654] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.503348 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet cd92a1a62ab24647bef7c9eefdf9c9d8 (table default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]) successfully deleted
I20260926 19:58:53.498106 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:53.501744 12759 raft_consensus.cc:3060] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:53.510035 12519 leader_election.cc:343] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:53.510761 12519 leader_election.cc:343] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:53.511348 12519 leader_election.cc:343] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:53.512339 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c5ef9e1bca427fb889cbb17f77f999" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.501485 13724 raft_consensus.cc:515] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.517110 12759 raft_consensus.cc:2468] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:53.517421 13724 leader_election.cc:290] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.518697 13013 tablet_bootstrap.cc:492] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:53.501467 13705 tablet.cc:1751] T 7ef773d1c6694c63a011819024451b02 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:53.519106 13013 ts_tablet_manager.cc:1403] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.000s
I20260926 19:58:53.517952 13724 consensus_queue.cc:1048] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:53.503720 13569 tablet_replica.cc:333] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:53.527314 13569 raft_consensus.cc:2243] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:53.527899 13569 raft_consensus.cc:2272] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:53.530185 13569 ts_tablet_manager.cc:1916] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:53.530158 13013 raft_consensus.cc:359] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.530791 13013 raft_consensus.cc:385] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:53.531038 13013 raft_consensus.cc:740] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:53.531596 13013 consensus_queue.cc:260] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.534654 13716 raft_consensus.cc:493] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.534698 12387 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_070eaadfa595480ca36eede1de4c5a1f" } new_table_name: "default.Table_a305dca9a6e944b995ff332a5ce55fdb" modify_external_catalogs: true
I20260926 19:58:53.535125 13716 raft_consensus.cc:515] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.537536 13716 leader_election.cc:290] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.539824 12394 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_cc8a261b07da4eb384b6ac40b7c5c5c3" } new_table_name: "default.Table_5b512970ec854f3b922daef16491e2e2" modify_external_catalogs: true
I20260926 19:58:53.546540 13725 tablet.cc:1751] T 7ef773d1c6694c63a011819024451b02 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:53.555699 13013 ts_tablet_manager.cc:1434] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260926 19:58:53.561303 13013 tablet_bootstrap.cc:492] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:53.572296 13711 raft_consensus.cc:493] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.572865 13711 raft_consensus.cc:515] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.574980 13711 leader_election.cc:290] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.581120 13013 tablet_bootstrap.cc:654] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.582196 13203 ts_tablet_manager.cc:1929] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260926 19:58:53.582824 12387 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13e75602e3334f599230eb0cfe64e33f consensus state has no leader
W20260926 19:58:53.586335 12394 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8f818570b3a24899ae3abe19012efb16 consensus state has no leader
I20260926 19:58:53.590150 13203 log.cc:1199] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/381320b4d2844e2289a5ae390e6b0503
I20260926 19:58:53.592377 13203 ts_tablet_manager.cc:1950] T 381320b4d2844e2289a5ae390e6b0503 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:53.592484 13716 raft_consensus.cc:493] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.593973 13716 raft_consensus.cc:515] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.595923 13716 leader_election.cc:290] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.593091 13724 raft_consensus.cc:493] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.556098 13012 tablet_bootstrap.cc:492] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:53.596608 13724 raft_consensus.cc:515] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.597293 13012 ts_tablet_manager.cc:1403] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.288s user 0.008s sys 0.002s
I20260926 19:58:53.598546 13403 raft_consensus.cc:2272] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:53.598721 13724 leader_election.cc:290] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.599813 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 216) took 2154 ms. Trace:
I20260926 19:58:53.599681 13012 raft_consensus.cc:359] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.600314 13012 raft_consensus.cc:385] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:53.600585 13012 raft_consensus.cc:740] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:53.603538 13715 raft_consensus.cc:493] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.603623 13633 raft_consensus.cc:515] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.604010 13715 raft_consensus.cc:515] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:53.605590 12696 consensus_peers.cc:597] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:53.607204 12919 raft_consensus.cc:3060] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:53.609200 13715 leader_election.cc:290] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:53.609436 13203 rpcz_store.cc:276] 0926 19:58:51.445189 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.445298 (+ 109us) service_pool.cc:224] Handling call
0926 19:58:53.121900 (+1676602us) ts_tablet_manager.cc:1151] Deleting tablet 381320b4d2844e2289a5ae390e6b0503
0926 19:58:53.121937 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.183076 (+ 61139us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b2400002a38. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a1e5688a1 00007f9a1e56686e 00007f9a1e564665 00007f9a1e723888 00007f9a1e723acd 00007f9a1e723180 00000000005112e5 0000000000511289 00000000005c0508 00007f9a25646a0a 00007f9a256469ce
0926 19:58:53.476813 (+293737us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.582064 (+105251us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.599752 (+ 17688us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7469,"spinlock_wait_cycles":112518912,"tablet-delete.queue_time_us":1527580}
I20260926 19:58:53.601179 13012 consensus_queue.cc:260] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.614624 13203 tablet_replica.cc:333] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:53.615275 13012 ts_tablet_manager.cc:1434] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260926 19:58:53.615587 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9277497973f14dc6a9ebfdb45308c646"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:53.616694 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 381320b4d2844e2289a5ae390e6b0503 (table default.Table_a8870fb0212e43b788b3fced198aed4e [id=2206b9a20656421f8bbd96810835c982]) successfully deleted
I20260926 19:58:53.618289 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c8916d88dd48998eab514bd3c538b9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:53.618242 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_785a1ec64e194c679e03b352eb694de4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:53.618976 12589 raft_consensus.cc:2393] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in this term.
I20260926 19:58:53.619630 13013 tablet_bootstrap.cc:492] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
W20260926 19:58:53.619768 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_785a1ec64e194c679e03b352eb694de4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.620014 13013 ts_tablet_manager.cc:1403] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.000s
I20260926 19:58:53.620424 13203 raft_consensus.cc:2243] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:53.620885 13203 raft_consensus.cc:2272] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:53.628346 13203 ts_tablet_manager.cc:1916] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:53.629845 13715 consensus_queue.cc:1048] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:53.631378 13633 leader_election.cc:290] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.632138 12921 raft_consensus.cc:2393] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f6e9d6439fd54b28925a68a6145d3dee in current term 1: Already voted for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in this term.
I20260926 19:58:53.638170 13130 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:58:53.638518 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_4ceba47368094646b52f05a4bd441a64" } modify_external_catalogs: true
I20260926 19:58:53.639376 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_4ceba47368094646b52f05a4bd441a64" } modify_external_catalogs: true
I20260926 19:58:53.640461 13130 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:58:53.641110 13130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:58:53.622269 13013 raft_consensus.cc:359] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:53.643548 13012 tablet_bootstrap.cc:492] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:53.643890 13013 raft_consensus.cc:385] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:53.644210 13013 raft_consensus.cc:740] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:53.644958 13013 consensus_queue.cc:260] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:53.647037 13013 ts_tablet_manager.cc:1434] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
W20260926 19:58:53.648711 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9277497973f14dc6a9ebfdb45308c646 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.655786 13092 tablet_bootstrap.cc:654] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.657693 13013 tablet_bootstrap.cc:492] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:53.663295 13013 tablet_bootstrap.cc:654] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.678418 13013 tablet_bootstrap.cc:492] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
W20260926 19:58:53.683396 12519 leader_election.cc:343] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:53.703928 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a95683f794924e4fba011142bea60695"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:53.705552 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a95683f794924e4fba011142bea60695 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.684497 13724 raft_consensus.cc:493] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.685077 13013 ts_tablet_manager.cc:1403] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.000s
I20260926 19:58:53.713863 13724 raft_consensus.cc:515] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.715761 13724 leader_election.cc:290] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.716068 13013 raft_consensus.cc:359] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.716676 13013 raft_consensus.cc:385] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:58:53.716691 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 47989.1us
I20260926 19:58:53.716923 13013 raft_consensus.cc:740] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:53.717290 12696 leader_election.cc:343] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Not found: Tablet not found: 1455b99a65ff44ff933825196708f485
I20260926 19:58:53.717648 12696 leader_election.cc:304] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:58:53.718178 12696 raft_consensus.cc:2686] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 19:58:53.719798 13679 consensus_queue.cc:1048] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:53.668028 13403 ts_tablet_manager.cc:1916] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:53.722813 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.669133 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c8916d88dd48998eab514bd3c538b9" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:53.674031 13727 tablet.cc:1751] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:53.686124 12898 tablet_service.cc:1558] Processing DeleteTablet for tablet fb593ce2cd4147febebfe442de9fdd28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:53.730690 13012 tablet_bootstrap.cc:654] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.717478 13013 consensus_queue.cc:260] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.719627 12519 leader_election.cc:304] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:53.742110 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_7201d8013a434923a470d52f3d321e05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:53.742444 12696 leader_election.cc:304] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:53.743131 13724 raft_consensus.cc:2804] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:53.743548 13724 raft_consensus.cc:697] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
W20260926 19:58:53.743593 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7201d8013a434923a470d52f3d321e05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.743642 13715 raft_consensus.cc:3060] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:53.744333 13724 consensus_queue.cc:237] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.746207 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b734f0d33342d3be8b1827bcd5dd14" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.747601 12757 raft_consensus.cc:2468] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:53.748757 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "791b4ef9be004d7ead26591c50c6f405" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.751298 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89c8cbb097a644218e0ca033a6fdb4a2" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.740552 13013 ts_tablet_manager.cc:1434] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.027s user 0.003s sys 0.002s
I20260926 19:58:53.760391 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16df0828847d4e7db5702d84082fa402" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
W20260926 19:58:53.761336 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 456187us
I20260926 19:58:53.763140 13203 ts_tablet_manager.cc:1929] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:53.764664 13203 log.cc:1199] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/16df0828847d4e7db5702d84082fa402
I20260926 19:58:53.764551 12521 leader_election.cc:304] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:53.765388 13716 raft_consensus.cc:2749] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:53.765594 13203 ts_tablet_manager.cc:1950] T 16df0828847d4e7db5702d84082fa402 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:53.766040 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:53.766444 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 217) took 2257 ms. Trace:
I20260926 19:58:53.766856 12593 raft_consensus.cc:2393] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:53.767385 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:53.768165 13013 tablet_bootstrap.cc:492] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:53.768253 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16df0828847d4e7db5702d84082fa402" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.766745 13203 rpcz_store.cc:276] 0926 19:58:51.508728 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.508849 (+ 121us) service_pool.cc:224] Handling call
0926 19:58:53.614478 (+2105629us) ts_tablet_manager.cc:1151] Deleting tablet 16df0828847d4e7db5702d84082fa402
0926 19:58:53.614516 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.683641 (+ 69125us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.763026 (+ 79385us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.766397 (+ 3371us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2018214}
I20260926 19:58:53.769742 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 16df0828847d4e7db5702d84082fa402 (table default.Table_29ce86fb5af04ec4b8411a4f3135b79d [id=74eae66d927e413a9457c4c89cff78ce]) successfully deleted
W20260926 19:58:53.769917 12519 leader_election.cc:343] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:53.770699 12519 leader_election.cc:304] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:53.771548 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acdb4bd8cd58444cb1d6d4b5610de20d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:53.788938 13705 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:58:53.804764 13715 raft_consensus.cc:2749] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:53.807227 13724 raft_consensus.cc:2804] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:53.807569 13724 raft_consensus.cc:493] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:53.807910 13724 raft_consensus.cc:3060] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:53.809826 12519 leader_election.cc:343] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:53.810431 12519 leader_election.cc:343] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:53.810938 12519 leader_election.cc:343] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:53.770967 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 218) took 2254 ms. Trace:
I20260926 19:58:53.817569 13203 rpcz_store.cc:276] 0926 19:58:51.516201 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.516310 (+ 109us) service_pool.cc:224] Handling call
0926 19:58:53.770732 (+2254422us) ts_tablet_manager.cc:1151] Deleting tablet 64149f9e4d384ddbb922291ac7de35b7
0926 19:58:53.770773 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.770907 (+ 134us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2174135}
W20260926 19:58:53.819450 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 64149f9e4d384ddbb922291ac7de35b7 because the tablet was not found. No further retry: Not found: Tablet not found: 64149f9e4d384ddbb922291ac7de35b7
I20260926 19:58:53.819485 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 223) took 2164 ms. Trace:
I20260926 19:58:53.819823 13203 rpcz_store.cc:276] 0926 19:58:51.655065 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.655169 (+ 104us) service_pool.cc:224] Handling call
0926 19:58:53.818750 (+2163581us) ts_tablet_manager.cc:1151] Deleting tablet 3066a65ce44b4387a6f96124d9f476fb
0926 19:58:53.818792 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.819036 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2163048}
W20260926 19:58:53.820987 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 3066a65ce44b4387a6f96124d9f476fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3066a65ce44b4387a6f96124d9f476fb already in progress: creating tablet
I20260926 19:58:53.821235 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 231) took 1853 ms. Trace:
I20260926 19:58:53.821530 13203 rpcz_store.cc:276] 0926 19:58:51.967626 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.967775 (+ 149us) service_pool.cc:224] Handling call
0926 19:58:53.820933 (+1853158us) ts_tablet_manager.cc:1151] Deleting tablet 6e20b1a22af944a0aad0fee5fc835f6e
0926 19:58:53.820971 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.821190 (+ 219us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1852612}
I20260926 19:58:53.822993 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 233) took 1285 ms. Trace:
I20260926 19:58:53.823493 12388 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 8ff071a16f1244cebc8c93e5508280a3 (table default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7 [id=e452740c55c345a9ab7445009c7a71cb]). Expected version 2 got 0
I20260926 19:58:53.823305 13203 rpcz_store.cc:276] 0926 19:58:52.537958 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.538060 (+ 102us) service_pool.cc:224] Handling call
0926 19:58:53.822641 (+1284581us) ts_tablet_manager.cc:1151] Deleting tablet 56fd4c88f9304f19883c3e5da8f937b5
0926 19:58:53.822682 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.822945 (+ 263us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1284064}
I20260926 19:58:53.824064 12388 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:53.824909 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 234) took 1191 ms. Trace:
W20260926 19:58:53.824262 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 6e20b1a22af944a0aad0fee5fc835f6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e20b1a22af944a0aad0fee5fc835f6e already in progress: creating tablet
I20260926 19:58:53.820140 13569 ts_tablet_manager.cc:1929] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:53.825181 13203 rpcz_store.cc:276] 0926 19:58:52.633409 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.669618 (+ 36209us) service_pool.cc:224] Handling call
0926 19:58:53.824613 (+1154995us) ts_tablet_manager.cc:1151] Deleting tablet 51f5d8149d474ab1b52d53fc0b490032
0926 19:58:53.824649 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.824868 (+ 219us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1154479}
W20260926 19:58:53.825831 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 56fd4c88f9304f19883c3e5da8f937b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56fd4c88f9304f19883c3e5da8f937b5 already in progress: creating tablet
W20260926 19:58:53.826421 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 51f5d8149d474ab1b52d53fc0b490032 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51f5d8149d474ab1b52d53fc0b490032 already in progress: creating tablet
I20260926 19:58:53.826911 13569 log.cc:1199] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/1455b99a65ff44ff933825196708f485
I20260926 19:58:53.828296 13569 ts_tablet_manager.cc:1950] T 1455b99a65ff44ff933825196708f485 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
W20260926 19:58:53.779613 13741 log.cc:927] Time spent T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: Append to log took a long time: real 0.067s user 0.001s sys 0.000s
I20260926 19:58:53.824600 12388 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:53.840677 12564 tablet_service.cc:1511] Processing CreateTablet for tablet f98661b53a4742fda386590598f3ac1a (DEFAULT_TABLE table=default.Table_9220288c84b847fea7df2fc881557a09 [id=257243f16bce441e8a66610254a98d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.841650 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f98661b53a4742fda386590598f3ac1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.843642 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:53.846081 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:53.843757 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 0720cd9d2a9a4f978d9e29a193d4a77a (DEFAULT_TABLE table=default.Table_f049d03e319247ad9be123472530cea4 [id=b6618e509b97420a870765f4609da4db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.848732 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0720cd9d2a9a4f978d9e29a193d4a77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.831152 13012 tablet_bootstrap.cc:492] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:53.850104 13012 ts_tablet_manager.cc:1403] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.207s user 0.011s sys 0.002s
I20260926 19:58:53.856403 12389 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 7ef773d1c6694c63a011819024451b02 (table default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa]). Expected version 1 got 0
I20260926 19:58:53.857179 12389 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:53.858489 12389 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:53.864377 12735 tablet_service.cc:1511] Processing CreateTablet for tablet 0720cd9d2a9a4f978d9e29a193d4a77a (DEFAULT_TABLE table=default.Table_f049d03e319247ad9be123472530cea4 [id=b6618e509b97420a870765f4609da4db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.864471 12737 tablet_service.cc:1511] Processing CreateTablet for tablet f98661b53a4742fda386590598f3ac1a (DEFAULT_TABLE table=default.Table_9220288c84b847fea7df2fc881557a09 [id=257243f16bce441e8a66610254a98d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.859387 12389 catalog_manager.cc:5838] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 1, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 1 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:53.866323 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 66c8916d88dd48998eab514bd3c538b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 700d05149a934d5e8bec9cc41873ad3e) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:53.867209 12389 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 8ff071a16f1244cebc8c93e5508280a3 (table default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7 [id=e452740c55c345a9ab7445009c7a71cb]). Expected version 2 got 0
I20260926 19:58:53.868136 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0720cd9d2a9a4f978d9e29a193d4a77a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:53.870383 12382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.253:37913, user_credentials={real_user=slave}} blocked reactor thread for 80409.3us
I20260926 19:58:53.872027 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f98661b53a4742fda386590598f3ac1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.873816 12562 tablet_service.cc:1511] Processing CreateTablet for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 (DEFAULT_TABLE table=default.Table_01a9ebf62e394638933953cc6654b838 [id=0ddaa6e4a56742f79c1e02c0df0e4b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.874812 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfea8dbdd1ba49a5bd9d17652eb271c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.876456 12741 tablet_service.cc:1511] Processing CreateTablet for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 (DEFAULT_TABLE table=default.Table_01a9ebf62e394638933953cc6654b838 [id=0ddaa6e4a56742f79c1e02c0df0e4b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.877419 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfea8dbdd1ba49a5bd9d17652eb271c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.879110 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 6c05c67d89814ca6bc906f4b683523b0 (DEFAULT_TABLE table=default.Table_91489cb29847444d9e79a0587e38a517 [id=f4965bb62a8a4413bb1cd2aeefa01a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.879995 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c05c67d89814ca6bc906f4b683523b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.882185 12560 tablet_service.cc:1511] Processing CreateTablet for tablet cb6c12fcb5444809bfe3fc7dca9bdec4 (DEFAULT_TABLE table=default.Table_b25cee2337e04223b7c489d0cea2db58 [id=4f3ed4aef41a414c8c06a537a0f7ad3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.883059 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb6c12fcb5444809bfe3fc7dca9bdec4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:53.883136 12382 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet d4434cb08d0f41fd96283de49c53b3e8 (table default.Table_17815f4814b9429da94beee1f7a60322 [id=fcf0c9e6dcd647d09ad31bbc20abd068], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet d4434cb08d0f41fd96283de49c53b3e8 consensus state has no leader
I20260926 19:58:53.886950 12738 tablet_service.cc:1511] Processing CreateTablet for tablet cb6c12fcb5444809bfe3fc7dca9bdec4 (DEFAULT_TABLE table=default.Table_b25cee2337e04223b7c489d0cea2db58 [id=4f3ed4aef41a414c8c06a537a0f7ad3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.887869 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb6c12fcb5444809bfe3fc7dca9bdec4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:53.862454 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 50718.8us
I20260926 19:58:53.890100 12561 tablet_service.cc:1511] Processing CreateTablet for tablet e9c7b63db44742108dc132df5089e0d9 (DEFAULT_TABLE table=default.Table_408479a582a04c8b829f7474e9cd00ee [id=c4265098bdb34d0ab60545dd2293ba84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.891021 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9c7b63db44742108dc132df5089e0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.894177 12740 tablet_service.cc:1511] Processing CreateTablet for tablet e9c7b63db44742108dc132df5089e0d9 (DEFAULT_TABLE table=default.Table_408479a582a04c8b829f7474e9cd00ee [id=c4265098bdb34d0ab60545dd2293ba84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.894389 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_408479a582a04c8b829f7474e9cd00ee" } new_table_name: "default.Table_d6d28646357a4b26ba16bbb1a7eb01d9" modify_external_catalogs: true
I20260926 19:58:53.894477 12558 tablet_service.cc:1511] Processing CreateTablet for tablet 90b7a9622b0f412b9a88aa84419ad0d7 (DEFAULT_TABLE table=default.Table_66bb1825f60742cc90be30615c83724a [id=b1b956e2eb0740bab149e499b6707263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.895079 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9c7b63db44742108dc132df5089e0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.895452 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90b7a9622b0f412b9a88aa84419ad0d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.898435 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 6c05c67d89814ca6bc906f4b683523b0 (DEFAULT_TABLE table=default.Table_91489cb29847444d9e79a0587e38a517 [id=f4965bb62a8a4413bb1cd2aeefa01a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.898701 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_46c845de9a6f4e368a893c401a7f43d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:53.899936 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c05c67d89814ca6bc906f4b683523b0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:53.900053 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46c845de9a6f4e368a893c401a7f43d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.900105 13724 raft_consensus.cc:515] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.907332 12734 tablet_service.cc:1511] Processing CreateTablet for tablet 90b7a9622b0f412b9a88aa84419ad0d7 (DEFAULT_TABLE table=default.Table_66bb1825f60742cc90be30615c83724a [id=b1b956e2eb0740bab149e499b6707263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.908411 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90b7a9622b0f412b9a88aa84419ad0d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.914167 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_d4eea629040e41128693ff3cab8caa65" } new_table_name: "default.Table_42e2147189dd490180e130039faf6cdc" modify_external_catalogs: true
I20260926 19:58:53.799993 13403 ts_tablet_manager.cc:1929] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:53.923916 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 81918e7e00f84a0ab8e19963a9d20332 (DEFAULT_TABLE table=default.Table_de6c58034b1949dfa5e5f83bd07db6d7 [id=f91be1fbbd294acf88688109480b4be2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.924866 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81918e7e00f84a0ab8e19963a9d20332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.924938 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 81918e7e00f84a0ab8e19963a9d20332 (DEFAULT_TABLE table=default.Table_de6c58034b1949dfa5e5f83bd07db6d7 [id=f91be1fbbd294acf88688109480b4be2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.925877 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81918e7e00f84a0ab8e19963a9d20332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.927603 12568 tablet_service.cc:1511] Processing CreateTablet for tablet fcd4094760e3457baab62ea4b10b6cc6 (DEFAULT_TABLE table=default.Table_8e95dc18c4dd485bb6eae37e9dccaf1f [id=1a29e65f818e49e6af04dae37cd39889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.928483 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd4094760e3457baab62ea4b10b6cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.935559 12391 master_service.cc:960] ReplaceTablet: received request to replace tablet cd86f01d82d041f4852f83f2547e697f from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:53.943439 13013 tablet_bootstrap.cc:654] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:53.945140 13748 tablet.cc:1751] T 7ef773d1c6694c63a011819024451b02 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:53.945685 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 0720cd9d2a9a4f978d9e29a193d4a77a (DEFAULT_TABLE table=default.Table_f049d03e319247ad9be123472530cea4 [id=b6618e509b97420a870765f4609da4db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.947337 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0720cd9d2a9a4f978d9e29a193d4a77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.947220 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd86f01d82d041f4852f83f2547e697f" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:53.948263 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:53.948886 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bdb479c5384d0891db66943ef263b2" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:53.953630 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c5ef9e1bca427fb889cbb17f77f999" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:53.954327 13720 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:33594 (request call id 224) took 1273 ms. Trace:
I20260926 19:58:53.956008 12898 tablet_service.cc:1511] Processing CreateTablet for tablet f98661b53a4742fda386590598f3ac1a (DEFAULT_TABLE table=default.Table_9220288c84b847fea7df2fc881557a09 [id=257243f16bce441e8a66610254a98d36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.956933 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f98661b53a4742fda386590598f3ac1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.923730 13403 log.cc:1199] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/77b319b685344a848462b78617eb6c22
I20260926 19:58:53.962272 13724 leader_election.cc:290] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:53.962163 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_4d7aa0cb1d464b23925eff429da34da9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:53.963115 12741 tablet_service.cc:1511] Processing CreateTablet for tablet fcd4094760e3457baab62ea4b10b6cc6 (DEFAULT_TABLE table=default.Table_8e95dc18c4dd485bb6eae37e9dccaf1f [id=1a29e65f818e49e6af04dae37cd39889]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:53.963681 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d7aa0cb1d464b23925eff429da34da9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:53.964033 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd4094760e3457baab62ea4b10b6cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.917941 13711 raft_consensus.cc:493] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:53.967821 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b734f0d33342d3be8b1827bcd5dd14" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:53.968394 12759 raft_consensus.cc:3060] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:53.969861 12557 tablet_service.cc:1511] Processing CreateTablet for tablet deb7ea3bde5e4464a48b6e2843599f7d (DEFAULT_TABLE table=default.Table_aa7b619547ec4df09f564d512fb394c7 [id=d11641721bb24f01a669d04212f81cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.970748 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b734f0d33342d3be8b1827bcd5dd14" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:53.920708 13718 consensus_queue.cc:1048] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260926 19:58:53.975440 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:53.975517 12739 tablet_service.cc:1511] Processing CreateTablet for tablet deb7ea3bde5e4464a48b6e2843599f7d (DEFAULT_TABLE table=default.Table_aa7b619547ec4df09f564d512fb394c7 [id=d11641721bb24f01a669d04212f81cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.975818 12759 raft_consensus.cc:2468] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:53.976488 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb7ea3bde5e4464a48b6e2843599f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.977633 12519 leader_election.cc:304] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:53.994320 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 9599f427a64f40d7b185b23a849b46f3 (DEFAULT_TABLE table=default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b [id=5639673ab1364e00a44bc7bce0428012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.995318 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9599f427a64f40d7b185b23a849b46f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.995297 12556 tablet_service.cc:1511] Processing CreateTablet for tablet 9599f427a64f40d7b185b23a849b46f3 (DEFAULT_TABLE table=default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b [id=5639673ab1364e00a44bc7bce0428012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:53.997684 12557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb7ea3bde5e4464a48b6e2843599f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.000453 13716 raft_consensus.cc:2804] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:54.000895 13716 raft_consensus.cc:697] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:54.001659 13716 consensus_queue.cc:237] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.012239 12731 tablet_service.cc:1511] Processing CreateTablet for tablet f5680b405ba94a419137a5559f1d26c4 (DEFAULT_TABLE table=default.Table_ba5394bd03c6428eb734bb5133243b18 [id=9f4495764cb94864b1098ec489559b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.022213 12568 tablet_service.cc:1511] Processing CreateTablet for tablet f5680b405ba94a419137a5559f1d26c4 (DEFAULT_TABLE table=default.Table_ba5394bd03c6428eb734bb5133243b18 [id=9f4495764cb94864b1098ec489559b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.023383 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5680b405ba94a419137a5559f1d26c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.025460 12556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9599f427a64f40d7b185b23a849b46f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:53.853147 13012 raft_consensus.cc:359] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.032495 12730 tablet_service.cc:1511] Processing CreateTablet for tablet d734112eca1249559d0ff3d123e82042 (DEFAULT_TABLE table=default.Table_174debbe544e47ccbcf8faef177fdd0b [id=1a610b5d22a14bd79b983901002df704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.032992 13012 raft_consensus.cc:385] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.033279 13012 raft_consensus.cc:740] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.033852 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 235) took 1356 ms. Trace:
I20260926 19:58:54.033895 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16897e7489e9445ba5389a4f3507a739" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:54.034042 13012 consensus_queue.cc:260] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.034435 12923 raft_consensus.cc:3060] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:54.034188 13203 rpcz_store.cc:276] 0926 19:58:52.677242 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.677343 (+ 101us) service_pool.cc:224] Handling call
0926 19:58:53.993885 (+1316542us) ts_tablet_manager.cc:1151] Deleting tablet 51f5d8149d474ab1b52d53fc0b490032
0926 19:58:54.024842 (+ 30957us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.033795 (+ 8953us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1316025}
W20260926 19:58:54.035399 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 51f5d8149d474ab1b52d53fc0b490032 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51f5d8149d474ab1b52d53fc0b490032 already in progress: creating tablet
I20260926 19:58:54.036484 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 236) took 1358 ms. Trace:
I20260926 19:58:54.036803 13203 rpcz_store.cc:276] 0926 19:58:52.678131 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.690007 (+ 11876us) service_pool.cc:224] Handling call
0926 19:58:54.036144 (+1346137us) ts_tablet_manager.cc:1151] Deleting tablet 5450b4cd35f04904845f1612fe58be52
0926 19:58:54.036186 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.036432 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1345615}
W20260926 19:58:54.037959 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 5450b4cd35f04904845f1612fe58be52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5450b4cd35f04904845f1612fe58be52 already in progress: creating tablet
W20260926 19:58:54.038501 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 60943.1us
I20260926 19:58:54.038758 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 6c05c67d89814ca6bc906f4b683523b0
I20260926 19:58:54.040673 13092 tablet_bootstrap.cc:492] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:54.041055 13092 ts_tablet_manager.cc:1403] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.601s user 0.009s sys 0.005s
I20260926 19:58:54.041024 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c05c67d89814ca6bc906f4b683523b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:54.032735 12917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acdb4bd8cd58444cb1d6d4b5610de20d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.044173 12904 tablet_service.cc:1511] Processing CreateTablet for tablet cb6c12fcb5444809bfe3fc7dca9bdec4 (DEFAULT_TABLE table=default.Table_b25cee2337e04223b7c489d0cea2db58 [id=4f3ed4aef41a414c8c06a537a0f7ad3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.042372 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 6c05c67d89814ca6bc906f4b683523b0 (DEFAULT_TABLE table=default.Table_91489cb29847444d9e79a0587e38a517 [id=f4965bb62a8a4413bb1cd2aeefa01a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.048454 12729 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c05c67d89814ca6bc906f4b683523b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:54.050473 13013 tablet_bootstrap.cc:492] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:54.050860 13013 ts_tablet_manager.cc:1403] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.292s user 0.009s sys 0.000s
I20260926 19:58:53.965633 13711 raft_consensus.cc:515] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.053103 12558 tablet_service.cc:1511] Processing CreateTablet for tablet d734112eca1249559d0ff3d123e82042 (DEFAULT_TABLE table=default.Table_174debbe544e47ccbcf8faef177fdd0b [id=1a610b5d22a14bd79b983901002df704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.054450 12895 tablet_service.cc:1511] Processing CreateTablet for tablet e9c7b63db44742108dc132df5089e0d9 (DEFAULT_TABLE table=default.Table_408479a582a04c8b829f7474e9cd00ee [id=c4265098bdb34d0ab60545dd2293ba84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.053167 13013 raft_consensus.cc:359] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.060361 13013 raft_consensus.cc:385] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.060711 13013 raft_consensus.cc:740] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.061880 12896 tablet_service.cc:1511] Processing CreateTablet for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 (DEFAULT_TABLE table=default.Table_01a9ebf62e394638933953cc6654b838 [id=0ddaa6e4a56742f79c1e02c0df0e4b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.062951 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfea8dbdd1ba49a5bd9d17652eb271c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.063287 12729 tablet_service.cc:1511] Processing CreateTablet for tablet f4ee22184d1240a2b62bcdddb596c985 (DEFAULT_TABLE table=default.Table_9cad47f6e7924243acf6f9c6b9a0573f [id=43dcb6dfaca94bccb4ac5d496d96bfe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.066413 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "791b4ef9be004d7ead26591c50c6f405" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.067036 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d734112eca1249559d0ff3d123e82042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.066922 13013 consensus_queue.cc:260] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:53.954704 13720 rpcz_store.cc:276] 0926 19:58:52.646199 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.646306 (+ 107us) service_pool.cc:224] Handling call
0926 19:58:53.917569 (+1271263us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:52.653666 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0926 19:58:52.653977 (+ 311us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0926 19:58:52.654009 (+ 32us) alter_schema_op.cc:66] Acquired schema lock
0926 19:58:52.654047 (+ 38us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0926 19:58:52.654151 (+ 104us) alter_schema_op.cc:128] START. Timestamp: 1790452732654089
0926 19:58:52.654172 (+ 21us) op_driver.cc:348] REPLICATION: starting
0926 19:58:52.654617 (+ 445us) log.cc:844] Serialized 173 byte log entry
0926 19:58:53.321208 (+666591us) op_driver.cc:464] REPLICATION: finished
0926 19:58:53.455230 (+134022us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0926 19:58:53.457249 (+ 2019us) log.cc:844] Serialized 5 byte log entry
0926 19:58:53.706594 (+249345us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.707424 (+ 830us) log.cc:844] Serialized 14 byte log entry
0926 19:58:53.830954 (+123530us) alter_schema_op.cc:73] Released schema lock
0926 19:58:53.876369 (+ 45415us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":133736,"prepare.queue_time_us":6627,"prepare.run_cpu_time_us":3614,"prepare.run_wall_time_us":159054,"raft.queue_time_us":1325,"raft.run_cpu_time_us":1937,"raft.run_wall_time_us":20862,"replication_time_us":666821,"thread_start_us":292875,"threads_started":2,"wal-append.queue_time_us":139973}]]}
I20260926 19:58:54.071094 12731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5680b405ba94a419137a5559f1d26c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.074209 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 245) took 1093 ms. Trace:
I20260926 19:58:54.074528 13203 rpcz_store.cc:276] 0926 19:58:52.980394 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.098571 (+118177us) service_pool.cc:224] Handling call
0926 19:58:54.059761 (+961190us) ts_tablet_manager.cc:1151] Deleting tablet 3215bbd5be56438491a73925ac9ccbe6
0926 19:58:54.073976 (+ 14215us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.074158 (+ 182us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":952336}
I20260926 19:58:54.048200 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89c8cbb097a644218e0ca033a6fdb4a2" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:54.075706 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 3215bbd5be56438491a73925ac9ccbe6 because the tablet was not found. No further retry: Not found: Tablet not found: 3215bbd5be56438491a73925ac9ccbe6
I20260926 19:58:54.076475 13203 tablet_replica.cc:333] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:54.078045 13013 ts_tablet_manager.cc:1434] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.027s user 0.007s sys 0.001s
I20260926 19:58:54.079205 13203 raft_consensus.cc:2243] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:54.079857 13203 raft_consensus.cc:2272] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:53.849488 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 208) took 2792 ms. Trace:
I20260926 19:58:54.082504 13203 ts_tablet_manager.cc:1916] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.082151 13569 rpcz_store.cc:276] 0926 19:58:51.057606 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.111661 (+ 54055us) service_pool.cc:224] Handling call
0926 19:58:53.503506 (+2391845us) ts_tablet_manager.cc:1151] Deleting tablet 1455b99a65ff44ff933825196708f485
0926 19:58:53.503551 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:53.629087 (+125536us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.819997 (+190910us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:53.849429 (+ 29432us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2391175}
I20260926 19:58:54.083787 13569 tablet_replica.cc:333] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:54.083918 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 1455b99a65ff44ff933825196708f485 (table default.Table_f1e412eb951f446990776744666978fc [id=bfb91cbd91534b4699fd4423b9923881]) successfully deleted
I20260926 19:58:54.086287 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 81918e7e00f84a0ab8e19963a9d20332 (DEFAULT_TABLE table=default.Table_de6c58034b1949dfa5e5f83bd07db6d7 [id=f91be1fbbd294acf88688109480b4be2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.086809 12562 tablet_service.cc:1511] Processing CreateTablet for tablet f4ee22184d1240a2b62bcdddb596c985 (DEFAULT_TABLE table=default.Table_9cad47f6e7924243acf6f9c6b9a0573f [id=43dcb6dfaca94bccb4ac5d496d96bfe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.088837 13013 tablet_bootstrap.cc:492] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:54.093605 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ee22184d1240a2b62bcdddb596c985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.099251 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 222e3d41fe404c0cb2921eebc94ec622 (DEFAULT_TABLE table=default.Table_27e7d0d0005441c8b6d63013d60972aa [id=bd098102c48c468d8089f64d3f3bd384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.100345 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 222e3d41fe404c0cb2921eebc94ec622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.102779 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 90b7a9622b0f412b9a88aa84419ad0d7 (DEFAULT_TABLE table=default.Table_66bb1825f60742cc90be30615c83724a [id=b1b956e2eb0740bab149e499b6707263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.104091 12924 raft_consensus.cc:2468] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 0.
I20260926 19:58:54.106089 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 13bb4e7eeef145fc85a5c412add99ce7
I20260926 19:58:54.107515 12393 catalog_manager.cc:6934] ReplaceTablet: tablet 13bb4e7eeef145fc85a5c412add99ce7 deleted and replaced by tablet 6a85852c08304e0fbbbcbb2c0d24b27e
I20260926 19:58:54.107580 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb6c12fcb5444809bfe3fc7dca9bdec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.109540 12894 tablet_service.cc:1511] Processing CreateTablet for tablet 9599f427a64f40d7b185b23a849b46f3 (DEFAULT_TABLE table=default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b [id=5639673ab1364e00a44bc7bce0428012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.112599 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 2ad76fc959e24545bdfad19f988fce93
I20260926 19:58:54.114707 13013 tablet_bootstrap.cc:654] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.114482 12893 tablet_service.cc:1511] Processing CreateTablet for tablet f5680b405ba94a419137a5559f1d26c4 (DEFAULT_TABLE table=default.Table_ba5394bd03c6428eb734bb5133243b18 [id=9f4495764cb94864b1098ec489559b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.118906 12728 tablet_service.cc:1558] Processing DeleteTablet for tablet 13bb4e7eeef145fc85a5c412add99ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a85852c08304e0fbbbcbb2c0d24b27e) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:54.120463 12728 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ad76fc959e24545bdfad19f988fce93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:54.120497 12857 leader_election.cc:343] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:54.127283 12521 leader_election.cc:343] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.127772 12521 leader_election.cc:304] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.128750 13724 raft_consensus.cc:2749] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:54.129802 12521 leader_election.cc:343] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.130156 12521 leader_election.cc:304] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.129153 12564 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ad76fc959e24545bdfad19f988fce93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:54.131258 13724 raft_consensus.cc:2749] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:54.132318 12521 leader_election.cc:343] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.132682 12521 leader_election.cc:304] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.133363 13013 tablet_bootstrap.cc:492] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:54.133646 13724 raft_consensus.cc:2749] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:54.133760 13013 ts_tablet_manager.cc:1403] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.045s user 0.010s sys 0.000s
W20260926 19:58:54.140671 12521 leader_election.cc:343] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:54.141446 12521 leader_election.cc:343] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.142252 12521 leader_election.cc:304] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.142856 13724 raft_consensus.cc:2749] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:54.142897 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_716c1e6e330044689f2657d0d570f0fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:54.144443 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_716c1e6e330044689f2657d0d570f0fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:54.145318 12892 tablet_service.cc:1511] Processing CreateTablet for tablet d734112eca1249559d0ff3d123e82042 (DEFAULT_TABLE table=default.Table_174debbe544e47ccbcf8faef177fdd0b [id=1a610b5d22a14bd79b983901002df704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.147008 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_81cb6f916c5d49a18d46e44cfe747ee0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:54.148473 13012 ts_tablet_manager.cc:1434] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.298s user 0.003s sys 0.002s
W20260926 19:58:54.148574 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_81cb6f916c5d49a18d46e44cfe747ee0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:54.148722 12897 tablet_service.cc:1511] Processing CreateTablet for tablet deb7ea3bde5e4464a48b6e2843599f7d (DEFAULT_TABLE table=default.Table_aa7b619547ec4df09f564d512fb394c7 [id=d11641721bb24f01a669d04212f81cac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.149443 12902 tablet_service.cc:1511] Processing CreateTablet for tablet fcd4094760e3457baab62ea4b10b6cc6 (DEFAULT_TABLE table=default.Table_8e95dc18c4dd485bb6eae37e9dccaf1f [id=1a29e65f818e49e6af04dae37cd39889]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.150413 13012 tablet_bootstrap.cc:492] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:54.152061 13203 ts_tablet_manager.cc:1929] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:54.152417 13203 log.cc:1199] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/77b319b685344a848462b78617eb6c22
I20260926 19:58:54.153074 13203 ts_tablet_manager.cc:1950] T 77b319b685344a848462b78617eb6c22 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:54.154258 13569 raft_consensus.cc:2243] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:54.154760 13569 raft_consensus.cc:2272] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.156603 13569 ts_tablet_manager.cc:1916] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.158406 13013 raft_consensus.cc:359] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.159011 13013 raft_consensus.cc:385] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.159277 13013 raft_consensus.cc:740] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.161914 13013 consensus_queue.cc:260] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.150327 12894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9599f427a64f40d7b185b23a849b46f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.165688 13013 ts_tablet_manager.cc:1434] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260926 19:58:54.159631 13403 ts_tablet_manager.cc:1950] T 77b319b685344a848462b78617eb6c22 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:54.166487 13013 tablet_bootstrap.cc:492] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:54.166108 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd4094760e3457baab62ea4b10b6cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.168284 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 222e3d41fe404c0cb2921eebc94ec622 (DEFAULT_TABLE table=default.Table_27e7d0d0005441c8b6d63013d60972aa [id=bd098102c48c468d8089f64d3f3bd384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.169309 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 222e3d41fe404c0cb2921eebc94ec622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.172683 12891 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c05c67d89814ca6bc906f4b683523b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:54.158931 13711 leader_election.cc:290] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:54.175782 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet deb7ea3bde5e4464a48b6e2843599f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.180181 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 246) took 1199 ms. Trace:
I20260926 19:58:54.174376 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b734f0d33342d3be8b1827bcd5dd14" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:54.182199 13203 rpcz_store.cc:276] 0926 19:58:52.980674 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.113994 (+133320us) service_pool.cc:224] Handling call
0926 19:58:54.076342 (+962348us) ts_tablet_manager.cc:1151] Deleting tablet 77b319b685344a848462b78617eb6c22
0926 19:58:54.076378 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.123374 (+ 46996us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.151949 (+ 28575us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.180113 (+ 28164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1794,"tablet-delete.queue_time_us":954171}
I20260926 19:58:54.119000 13720 tablet.cc:1751] T 53668c46bd3441ad936bf990b3192420 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:58:54.183583 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 249) took 1066 ms. Trace:
I20260926 19:58:54.183904 13203 rpcz_store.cc:276] 0926 19:58:53.116878 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.121900 (+ 5022us) service_pool.cc:224] Handling call
0926 19:58:54.183224 (+1061324us) ts_tablet_manager.cc:1151] Deleting tablet 8c7d0435f3564cb38b6cbca713009c2e
0926 19:58:54.183272 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.183520 (+ 248us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1060792}
I20260926 19:58:54.185607 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet 13bb4e7eeef145fc85a5c412add99ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a85852c08304e0fbbbcbb2c0d24b27e) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:54.186144 12891 tablet_service.cc:1511] Processing CreateTablet for tablet f4ee22184d1240a2b62bcdddb596c985 (DEFAULT_TABLE table=default.Table_9cad47f6e7924243acf6f9c6b9a0573f [id=43dcb6dfaca94bccb4ac5d496d96bfe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.186669 12730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d734112eca1249559d0ff3d123e82042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.190065 12729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ee22184d1240a2b62bcdddb596c985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.195732 13013 tablet_bootstrap.cc:654] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.205788 12891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ee22184d1240a2b62bcdddb596c985. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:54.209365 12389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 56) took 1023 ms (client timeout 1000 ms). Trace:
I20260926 19:58:54.213675 12893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5680b405ba94a419137a5559f1d26c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.215638 12895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9c7b63db44742108dc132df5089e0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.220171 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet cd86f01d82d041f4852f83f2547e697f with delete_type TABLET_DATA_DELETED (replaced by tablet 3488f0076b0f4ef19f7cf23a93cd20e3) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:54.220743 12556 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:44 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:54.220988 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:54.221727 12560 tablet_service.cc:1558] Processing DeleteTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:54.221932 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 248) took 1108 ms. Trace:
I20260926 19:58:54.222239 13203 rpcz_store.cc:276] 0926 19:58:53.116559 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.117893 (+ 1334us) service_pool.cc:224] Handling call
0926 19:58:54.194456 (+1076563us) ts_tablet_manager.cc:1151] Deleting tablet 5930402db43740b4b291c6c238ecf52c
0926 19:58:54.221641 (+ 27185us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.221883 (+ 242us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1039098}
I20260926 19:58:54.223186 13203 tablet_replica.cc:333] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:54.223883 13203 raft_consensus.cc:2243] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:54.224289 13203 raft_consensus.cc:2272] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.212119 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 8ff071a16f1244cebc8c93e5508280a3 (table default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7 [id=e452740c55c345a9ab7445009c7a71cb]). Expected version 2 got 0
I20260926 19:58:54.226166 13203 ts_tablet_manager.cc:1916] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.227638 12892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d734112eca1249559d0ff3d123e82042. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:54.222510 12389 rpcz_store.cc:269] 0926 19:58:53.185743 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.185864 (+ 121us) service_pool.cc:224] Handling call
0926 19:58:53.226594 (+ 40730us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:54.209291 (+982697us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:53.979887 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:53.980230 (+ 343us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:53.980252 (+ 22us) write_op.cc:435] Acquired schema lock
0926 19:58:53.980264 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:53.980475 (+ 211us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:53.980506 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:53.980530 (+ 24us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:53.980648 (+ 118us) tablet.cc:701] Row locks acquired
0926 19:58:53.980660 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:53.980746 (+ 86us) write_op.cc:270] Start()
0926 19:58:53.980816 (+ 70us) write_op.cc:276] Timestamp: P: 1790452733980686 usec, L: 0
0926 19:58:53.980832 (+ 16us) op_driver.cc:348] REPLICATION: starting
0926 19:58:53.981223 (+ 391us) log.cc:844] Serialized 536 byte log entry
0926 19:58:54.140277 (+159054us) op_driver.cc:464] REPLICATION: finished
0926 19:58:54.157723 (+ 17446us) write_op.cc:301] APPLY: starting
0926 19:58:54.157782 (+ 59us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:54.157877 (+ 95us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:54.157980 (+ 103us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:54.158162 (+ 182us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:54.158295 (+ 133us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:54.158333 (+ 38us) write_op.cc:312] APPLY: finished
0926 19:58:54.158493 (+ 160us) log.cc:844] Serialized 23 byte log entry
0926 19:58:54.177663 (+ 19170us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:54.177734 (+ 71us) write_op.cc:454] Released schema lock
0926 19:58:54.177769 (+ 35us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":70768000,"child_traces":[["op",{"apply.queue_time_us":16832,"num_ops":1,"prepare.queue_time_us":110237,"prepare.run_cpu_time_us":1550,"prepare.run_wall_time_us":1549,"raft.queue_time_us":102,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":11,"replication_time_us":159344}]]}
I20260926 19:58:54.221407 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 77b319b685344a848462b78617eb6c22 (table default.Table_a958288944144fbd9cc934add059cdb0 [id=1850b06b63d24da0b91ad1a54eae3dd9]) successfully deleted
W20260926 19:58:54.229852 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 8c7d0435f3564cb38b6cbca713009c2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c7d0435f3564cb38b6cbca713009c2e already in progress: creating tablet
W20260926 19:58:54.230423 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 5930402db43740b4b291c6c238ecf52c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5930402db43740b4b291c6c238ecf52c already in progress: creating tablet
I20260926 19:58:54.232734 12890 tablet_service.cc:1558] Processing DeleteTablet for tablet 13bb4e7eeef145fc85a5c412add99ce7 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a85852c08304e0fbbbcbb2c0d24b27e) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:54.234174 12890 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ad76fc959e24545bdfad19f988fce93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:54.234094 13725 tablet.cc:1751] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:54.243213 12890 tablet_service.cc:1511] Processing CreateTablet for tablet 222e3d41fe404c0cb2921eebc94ec622 (DEFAULT_TABLE table=default.Table_27e7d0d0005441c8b6d63013d60972aa [id=bd098102c48c468d8089f64d3f3bd384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:54.244515 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90b7a9622b0f412b9a88aa84419ad0d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.244355 12389 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_935be674b5a9419296be4ffa3464b082" } modify_external_catalogs: true
I20260926 19:58:54.245692 12389 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_935be674b5a9419296be4ffa3464b082" } modify_external_catalogs: true
I20260926 19:58:54.248567 13549 raft_consensus.cc:493] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:54.249030 13549 raft_consensus.cc:515] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:54.248503 13705 tablet.cc:1751] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:54.251024 13549 leader_election.cc:290] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:54.252389 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f268b4a1f7423f917731c6b62aab3e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:54.253749 12519 leader_election.cc:343] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.257925 13549 consensus_queue.cc:1048] T 8ff071a16f1244cebc8c93e5508280a3 P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:54.259838 12521 leader_election.cc:343] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.260217 12521 leader_election.cc:304] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.263595 13012 tablet_bootstrap.cc:654] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.269960 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:54.270525 12593 raft_consensus.cc:2468] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:54.273978 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c05c67d89814ca6bc906f4b683523b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.245419 13092 raft_consensus.cc:359] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.275002 13203 ts_tablet_manager.cc:1929] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:54.275218 13092 raft_consensus.cc:385] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.275391 13203 log.cc:1199] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/66c8916d88dd48998eab514bd3c538b9
I20260926 19:58:54.275511 13092 raft_consensus.cc:740] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.275990 13203 ts_tablet_manager.cc:1950] T 66c8916d88dd48998eab514bd3c538b9 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:54.276580 12890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 222e3d41fe404c0cb2921eebc94ec622. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.281886 13403 tablet_replica.cc:333] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:54.282744 13403 raft_consensus.cc:2243] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
W20260926 19:58:54.291054 12858 leader_election.cc:343] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: Tablet not RUNNING: STOPPED
I20260926 19:58:54.292351 13403 pending_rounds.cc:70] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Trying to abort 1 pending ops.
I20260926 19:58:54.294330 12923 raft_consensus.cc:2468] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:54.303071 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:54.315976 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81918e7e00f84a0ab8e19963a9d20332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:54.291842 12858 leader_election.cc:304] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:54.317842 13724 raft_consensus.cc:493] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:54.318285 13724 raft_consensus.cc:515] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.318676 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 252) took 1142 ms. Trace:
I20260926 19:58:54.319018 13203 rpcz_store.cc:276] 0926 19:58:53.176588 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.176712 (+ 124us) service_pool.cc:224] Handling call
0926 19:58:54.223034 (+1046322us) ts_tablet_manager.cc:1151] Deleting tablet 66c8916d88dd48998eab514bd3c538b9
0926 19:58:54.223089 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.232822 (+ 9733us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.274883 (+ 42061us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.318619 (+ 43736us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40850,"tablet-delete.queue_time_us":1045511}
I20260926 19:58:54.320164 13724 leader_election.cc:290] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:54.320425 13203 tablet_replica.cc:333] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:54.320660 13549 raft_consensus.cc:2749] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:54.321162 13203 raft_consensus.cc:2243] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:54.321823 13203 pending_rounds.cc:70] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:54.322032 13203 pending_rounds.cc:77] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694398479757312 op_type: NO_OP noop_request { }
I20260926 19:58:54.322212 12391 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a29b5ffc06a5457db50ba2559768ba53" } modify_external_catalogs: true
I20260926 19:58:54.322264 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0583424ef64d9ea3beeb1c01ceafba" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:54.322543 13203 raft_consensus.cc:2889] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:54.322278 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 66c8916d88dd48998eab514bd3c538b9 (table default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]) successfully deleted
I20260926 19:58:54.322851 13203 raft_consensus.cc:2272] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.322724 12391 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_a29b5ffc06a5457db50ba2559768ba53" } modify_external_catalogs: true
W20260926 19:58:54.323546 12519 leader_election.cc:343] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.324995 13203 ts_tablet_manager.cc:1916] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:54.324209 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 64401.2us
W20260926 19:58:54.338511 12521 leader_election.cc:343] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.338904 12521 leader_election.cc:304] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.344573 13012 tablet_bootstrap.cc:492] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:54.345063 13012 ts_tablet_manager.cc:1403] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.196s user 0.007s sys 0.002s
I20260926 19:58:54.347540 13012 raft_consensus.cc:359] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.348147 13012 raft_consensus.cc:385] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.348462 13012 raft_consensus.cc:740] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.355484 13569 ts_tablet_manager.cc:1929] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:58:54.356649 13203 ts_tablet_manager.cc:1929] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:54.356992 13203 log.cc:1199] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/c5b734f0d33342d3be8b1827bcd5dd14
I20260926 19:58:54.357709 13203 ts_tablet_manager.cc:1950] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:54.358433 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 250) took 1182 ms. Trace:
I20260926 19:58:54.358721 13203 rpcz_store.cc:276] 0926 19:58:53.175990 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.177631 (+ 1641us) service_pool.cc:224] Handling call
0926 19:58:54.320279 (+1142648us) ts_tablet_manager.cc:1151] Deleting tablet c5b734f0d33342d3be8b1827bcd5dd14
0926 19:58:54.320322 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.331575 (+ 11253us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.356515 (+ 24940us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.358384 (+ 1869us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1142090}
I20260926 19:58:54.359865 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet c5b734f0d33342d3be8b1827bcd5dd14 (table default.Table_2d039187041243c8adf2651cf58f0b4a [id=dba288eafec847baaf3732ce6d4292b5]) successfully deleted
W20260926 19:58:54.360370 12857 leader_election.cc:343] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:54.362229 12395 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e9c7b63db44742108dc132df5089e0d9 consensus state has no leader
I20260926 19:58:54.369625 13711 raft_consensus.cc:2749] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:54.370757 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 251) took 1194 ms. Trace:
I20260926 19:58:54.371068 13203 rpcz_store.cc:276] 0926 19:58:53.176252 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.180685 (+ 4433us) service_pool.cc:224] Handling call
0926 19:58:54.370513 (+1189828us) ts_tablet_manager.cc:1151] Deleting tablet 886ab0feae1e49629290bc892e518816
0926 19:58:54.370555 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.370703 (+ 148us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1189222}
I20260926 19:58:54.292584 13403 pending_rounds.cc:77] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694392705757184 op_type: NO_OP noop_request { }
I20260926 19:58:54.374436 13403 raft_consensus.cc:2889] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:54.374871 13403 raft_consensus.cc:2272] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.375741 13203 tablet_replica.cc:333] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:54.376497 13203 raft_consensus.cc:2243] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
W20260926 19:58:54.376510 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 85483us
I20260926 19:58:54.377311 12858 leader_election.cc:304] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:54.379369 13549 raft_consensus.cc:2749] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:54.380520 12521 leader_election.cc:304] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:54.383165 13569 log.cc:1199] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/381320b4d2844e2289a5ae390e6b0503
I20260926 19:58:54.384135 13569 ts_tablet_manager.cc:1950] T 381320b4d2844e2289a5ae390e6b0503 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:54.385792 13549 raft_consensus.cc:2804] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:54.386111 13549 raft_consensus.cc:493] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:54.377739 13012 consensus_queue.cc:260] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.386411 13549 raft_consensus.cc:3060] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:54.395946 13549 raft_consensus.cc:515] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:54.398275 12521 leader_election.cc:343] T c5b734f0d33342d3be8b1827bcd5dd14 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.399441 13599 raft_consensus.cc:493] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:54.399863 13599 raft_consensus.cc:515] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.401690 13203 pending_rounds.cc:70] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:54.401945 13203 pending_rounds.cc:77] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694397429522432 op_type: NO_OP noop_request { }
I20260926 19:58:54.402180 13599 leader_election.cc:290] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:54.402278 13203 raft_consensus.cc:2889] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:54.402590 13203 raft_consensus.cc:2272] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.402288 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:54.403496 12593 raft_consensus.cc:2393] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:54.411192 13012 ts_tablet_manager.cc:1434] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.066s user 0.006s sys 0.001s
I20260926 19:58:54.412185 13012 tablet_bootstrap.cc:492] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:54.419889 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 441b881f3660495cba9b94a8c7464e99 has no consensus state
I20260926 19:58:54.426448 13549 leader_election.cc:290] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:54.419866 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:54.437223 13013 tablet_bootstrap.cc:492] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:54.437165 12759 raft_consensus.cc:3060] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:54.437664 13013 ts_tablet_manager.cc:1403] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.271s user 0.014s sys 0.037s
I20260926 19:58:54.440747 13013 raft_consensus.cc:359] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:54.451507 12858 leader_election.cc:304] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9
I20260926 19:58:54.453816 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 77b319b685344a848462b78617eb6c22 (table default.Table_a958288944144fbd9cc934add059cdb0 [id=1850b06b63d24da0b91ad1a54eae3dd9]) successfully deleted
I20260926 19:58:54.457947 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 213) took 3002 ms. Trace:
I20260926 19:58:54.458393 13569 rpcz_store.cc:276] 0926 19:58:51.455782 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.456055 (+ 273us) service_pool.cc:224] Handling call
0926 19:58:54.083643 (+2627588us) ts_tablet_manager.cc:1151] Deleting tablet 381320b4d2844e2289a5ae390e6b0503
0926 19:58:54.083687 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.281930 (+198243us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.355367 (+ 73437us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.457883 (+102516us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69712,"tablet-delete.queue_time_us":2626650}
I20260926 19:58:54.459987 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 381320b4d2844e2289a5ae390e6b0503 (table default.Table_a8870fb0212e43b788b3fced198aed4e [id=2206b9a20656421f8bbd96810835c982]) successfully deleted
I20260926 19:58:54.461040 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 214) took 2935 ms. Trace:
I20260926 19:58:54.461390 13569 rpcz_store.cc:276] 0926 19:58:51.525885 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.526400 (+ 515us) service_pool.cc:224] Handling call
0926 19:58:54.460768 (+2934368us) ts_tablet_manager.cc:1151] Deleting tablet 64149f9e4d384ddbb922291ac7de35b7
0926 19:58:54.460814 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.460981 (+ 167us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2933407}
W20260926 19:58:54.462666 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 64149f9e4d384ddbb922291ac7de35b7 because the tablet was not found. No further retry: Not found: Tablet not found: 64149f9e4d384ddbb922291ac7de35b7
I20260926 19:58:54.467100 13012 tablet_bootstrap.cc:654] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.474416 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f268b4a1f7423f917731c6b62aab3e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.449142 13711 raft_consensus.cc:2749] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:54.462582 13629 raft_consensus.cc:2804] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:54.493036 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 215) took 2940 ms. Trace:
I20260926 19:58:54.493588 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.493376 13569 rpcz_store.cc:276] 0926 19:58:51.552556 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.552688 (+ 132us) service_pool.cc:224] Handling call
0926 19:58:54.492779 (+2940091us) ts_tablet_manager.cc:1151] Deleting tablet 886ab0feae1e49629290bc892e518816
0926 19:58:54.492819 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.492984 (+ 165us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2913604}
I20260926 19:58:54.495537 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 218) took 2850 ms. Trace:
I20260926 19:58:54.495839 13569 rpcz_store.cc:276] 0926 19:58:51.644883 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.644996 (+ 113us) service_pool.cc:224] Handling call
0926 19:58:54.495078 (+2850082us) ts_tablet_manager.cc:1151] Deleting tablet 3066a65ce44b4387a6f96124d9f476fb
0926 19:58:54.495116 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.495485 (+ 369us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2849559}
W20260926 19:58:54.497005 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 3066a65ce44b4387a6f96124d9f476fb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3066a65ce44b4387a6f96124d9f476fb already in progress: creating tablet
I20260926 19:58:54.498195 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 225) took 2752 ms. Trace:
I20260926 19:58:54.498486 13569 rpcz_store.cc:276] 0926 19:58:51.745831 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:51.745968 (+ 137us) service_pool.cc:224] Handling call
0926 19:58:54.497671 (+2751703us) ts_tablet_manager.cc:1151] Deleting tablet 6e20b1a22af944a0aad0fee5fc835f6e
0926 19:58:54.497721 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.498149 (+ 428us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2750566}
I20260926 19:58:54.452656 13718 consensus_queue.cc:1048] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260926 19:58:54.481243 12915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0583424ef64d9ea3beeb1c01ceafba" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.509379 13203 ts_tablet_manager.cc:1916] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.441419 13013 raft_consensus.cc:385] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.519670 13013 raft_consensus.cc:740] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.520324 13013 consensus_queue.cc:260] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:54.526964 13013 ts_tablet_manager.cc:1434] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.089s user 0.006s sys 0.000s
I20260926 19:58:54.528156 13013 tablet_bootstrap.cc:492] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
W20260926 19:58:54.530257 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 6e20b1a22af944a0aad0fee5fc835f6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6e20b1a22af944a0aad0fee5fc835f6e already in progress: creating tablet
I20260926 19:58:54.531277 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 228) took 2252 ms. Trace:
I20260926 19:58:54.531679 13569 rpcz_store.cc:276] 0926 19:58:52.278938 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.279070 (+ 132us) service_pool.cc:224] Handling call
0926 19:58:54.530940 (+2251870us) ts_tablet_manager.cc:1151] Deleting tablet 56fd4c88f9304f19883c3e5da8f937b5
0926 19:58:54.530981 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.531225 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2251339}
W20260926 19:58:54.532859 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 56fd4c88f9304f19883c3e5da8f937b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56fd4c88f9304f19883c3e5da8f937b5 already in progress: creating tablet
I20260926 19:58:54.511590 12759 raft_consensus.cc:2468] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:54.534371 12519 leader_election.cc:304] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:54.535036 13549 raft_consensus.cc:2804] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:54.535480 13403 ts_tablet_manager.cc:1916] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.535478 13549 raft_consensus.cc:697] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:54.536207 13549 consensus_queue.cc:237] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.488399 13629 raft_consensus.cc:493] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:54.542645 13629 raft_consensus.cc:3060] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:54.551371 13092 consensus_queue.cc:260] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.553112 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 230) took 1887 ms. Trace:
I20260926 19:58:54.553422 13569 rpcz_store.cc:276] 0926 19:58:52.666101 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.696726 (+ 30625us) service_pool.cc:224] Handling call
0926 19:58:54.543401 (+1846675us) ts_tablet_manager.cc:1151] Deleting tablet 51f5d8149d474ab1b52d53fc0b490032
0926 19:58:54.543443 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.553047 (+ 9604us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1834776}
W20260926 19:58:54.555469 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 51f5d8149d474ab1b52d53fc0b490032 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51f5d8149d474ab1b52d53fc0b490032 already in progress: creating tablet
I20260926 19:58:54.556150 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:54.556649 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 229) took 1905 ms. Trace:
I20260926 19:58:54.553375 13092 ts_tablet_manager.cc:1434] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.512s user 0.009s sys 0.000s
I20260926 19:58:54.561975 13013 tablet_bootstrap.cc:654] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:54.562704 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 85984.9us
W20260926 19:58:54.563588 12521 leader_election.cc:343] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.565804 12521 leader_election.cc:304] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.566506 13724 raft_consensus.cc:2749] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:54.568473 12521 leader_election.cc:343] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.571986 13012 tablet_bootstrap.cc:492] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:54.572430 13012 ts_tablet_manager.cc:1403] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.160s user 0.011s sys 0.001s
I20260926 19:58:54.556937 13569 rpcz_store.cc:276] 0926 19:58:52.651655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.734054 (+ 82399us) service_pool.cc:224] Handling call
0926 19:58:54.556265 (+1822211us) ts_tablet_manager.cc:1151] Deleting tablet 51f5d8149d474ab1b52d53fc0b490032
0926 19:58:54.556313 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.556598 (+ 285us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1821617}
I20260926 19:58:54.568909 12521 leader_election.cc:304] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.578964 13549 raft_consensus.cc:2749] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:54.559633 13092 ts_tablet_manager.cc:1467] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Tablet startup took 518ms
I20260926 19:58:54.591466 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 231) took 1913 ms. Trace:
I20260926 19:58:54.591907 13569 rpcz_store.cc:276] 0926 19:58:52.677737 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.700267 (+ 22530us) service_pool.cc:224] Handling call
0926 19:58:54.591115 (+1890848us) ts_tablet_manager.cc:1151] Deleting tablet 5450b4cd35f04904845f1612fe58be52
0926 19:58:54.591158 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.591411 (+ 253us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1846115}
I20260926 19:58:54.590260 13012 raft_consensus.cc:359] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.593961 13012 raft_consensus.cc:385] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.594769 13012 raft_consensus.cc:740] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.594446 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 239) took 1749 ms. Trace:
I20260926 19:58:54.596176 13012 consensus_queue.cc:260] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:54.584695 13092 ts_tablet_manager.cc:1469] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Trace:
0926 19:58:43.172993 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:43.173139 (+ 146us) service_pool.cc:224] Handling call
0926 19:58:43.174512 (+ 1373us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:43.174596 (+ 84us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:43.355190 (+180594us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:43.450951 (+ 95761us) tablet_replica.cc:206] Creating consensus instance
0926 19:58:43.483206 (+ 32255us) inbound_call.cc:177] Queueing success response
0926 19:58:53.345360 (+9862154us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0926 19:58:54.041285 (+695925us) ts_tablet_manager.cc:1435] Starting tablet replica
0926 19:58:54.181607 (+140322us) tablet_replica.cc:261] Starting instrumentation
0926 19:58:54.182075 (+ 468us) tablet_replica.cc:280] Starting consensus
0926 19:58:54.551107 (+369032us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b4c00002680. stack: 00007ffa5c912323 00007ffa5c912220 00007ffa5c4b6574 0000000000516964 00000000005168f9 00007ffa61704fe0 00007ffa61715fc9 00007ffa616db234 00007ffa617034ed 00007ffa61703153 00007ffa63aa194b 00007ffa63aa2475 00007ffa63a9d6f0 00007ffa63a9c400 00007ffa63eeed3e 00007ffa64b573ba
Metrics: {"raft.queue_time_us":6791,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":162,"spinlock_wait_cycles":19288448,"tablet-open.queue_time_us":9862177}
I20260926 19:58:54.606793 13203 ts_tablet_manager.cc:1929] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260926 19:58:54.609930 12695 consensus_peers.cc:597] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 16df0828847d4e7db5702d84082fa402. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:54.612882 13203 log.cc:1199] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/16897e7489e9445ba5389a4f3507a739
I20260926 19:58:54.595434 13569 rpcz_store.cc:276] 0926 19:58:52.844654 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.844789 (+ 135us) service_pool.cc:224] Handling call
0926 19:58:54.593507 (+1748718us) ts_tablet_manager.cc:1151] Deleting tablet 0fd184e6295d47aeba809be6aab9ce1c
0926 19:58:54.594238 (+ 731us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.594399 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1748093}
I20260926 19:58:54.614935 13569 tablet_replica.cc:333] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:54.615686 13569 raft_consensus.cc:2243] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:54.616600 13569 raft_consensus.cc:2272] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.618441 13569 ts_tablet_manager.cc:1916] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.620234 13203 ts_tablet_manager.cc:1950] T 16897e7489e9445ba5389a4f3507a739 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:54.622215 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 253) took 1420 ms. Trace:
I20260926 19:58:54.624619 13203 rpcz_store.cc:276] 0926 19:58:53.201405 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.201527 (+ 122us) service_pool.cc:224] Handling call
0926 19:58:54.375587 (+1174060us) ts_tablet_manager.cc:1151] Deleting tablet 16897e7489e9445ba5389a4f3507a739
0926 19:58:54.375630 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.536915 (+161285us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.606667 (+ 69752us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.622154 (+ 15487us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24429,"tablet-delete.queue_time_us":1167038}
I20260926 19:58:54.625355 13403 ts_tablet_manager.cc:1929] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:54.625717 13403 log.cc:1199] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/66c8916d88dd48998eab514bd3c538b9
I20260926 19:58:54.628477 13203 tablet_replica.cc:333] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:54.630180 13203 raft_consensus.cc:2243] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:54.633181 13203 raft_consensus.cc:2272] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.633821 13092 tablet_bootstrap.cc:492] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:54.637739 13203 ts_tablet_manager.cc:1916] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.645243 13403 ts_tablet_manager.cc:1950] T 66c8916d88dd48998eab514bd3c538b9 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:54.620085 13012 ts_tablet_manager.cc:1434] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.047s user 0.006s sys 0.001s
I20260926 19:58:54.651711 13549 raft_consensus.cc:493] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:54.652244 13549 raft_consensus.cc:515] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.654915 13724 raft_consensus.cc:493] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:54.655483 13724 raft_consensus.cc:515] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.656752 13012 tablet_bootstrap.cc:492] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:54.658011 13724 leader_election.cc:290] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:54.659381 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2d4d45fdeb4802ac261b6c8de91c3c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:54.661059 12519 leader_election.cc:343] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.664359 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b32f8b0d480547a58ba5c6fed6483a74" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:54.666038 12519 leader_election.cc:343] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.666662 13549 leader_election.cc:290] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:54.668318 13013 tablet_bootstrap.cc:492] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:54.668156 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b32f8b0d480547a58ba5c6fed6483a74" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.668716 13013 ts_tablet_manager.cc:1403] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.141s user 0.019s sys 0.001s
I20260926 19:58:54.609827 13629 raft_consensus.cc:515] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.672446 13629 leader_election.cc:290] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:54.676441 12521 leader_election.cc:343] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.676910 12521 leader_election.cc:304] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.683647 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:54.684878 12759 raft_consensus.cc:2393] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:54.686215 13012 tablet_bootstrap.cc:654] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.689002 12918 raft_consensus.cc:2393] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in this term.
I20260926 19:58:54.691212 13549 raft_consensus.cc:2749] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:54.695065 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 51f5d8149d474ab1b52d53fc0b490032 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51f5d8149d474ab1b52d53fc0b490032 already in progress: creating tablet
W20260926 19:58:54.696307 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 5450b4cd35f04904845f1612fe58be52 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5450b4cd35f04904845f1612fe58be52 already in progress: creating tablet
I20260926 19:58:54.700217 12696 leader_election.cc:304] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:54.701134 13715 raft_consensus.cc:2749] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:54.704339 13715 raft_consensus.cc:493] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:54.704828 13715 raft_consensus.cc:515] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.705572 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2d4d45fdeb4802ac261b6c8de91c3c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.705689 13013 raft_consensus.cc:359] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.706300 13013 raft_consensus.cc:385] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:58:54.697305 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 0fd184e6295d47aeba809be6aab9ce1c because the tablet was not found. No further retry: Not found: Tablet not found: 0fd184e6295d47aeba809be6aab9ce1c
I20260926 19:58:54.706563 13013 raft_consensus.cc:740] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.707597 13715 leader_election.cc:290] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:54.709056 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bdb479c5384d0891db66943ef263b2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:54.709779 12593 raft_consensus.cc:2468] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
W20260926 19:58:54.711246 12521 leader_election.cc:343] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.711119 12695 leader_election.cc:304] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:54.711603 12521 leader_election.cc:304] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.712425 13715 raft_consensus.cc:2804] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:54.712828 13715 raft_consensus.cc:493] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:54.713202 13715 raft_consensus.cc:3060] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:54.713104 13549 raft_consensus.cc:2749] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:54.713723 13013 consensus_queue.cc:260] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:54.714874 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 256184us
I20260926 19:58:54.715710 13013 ts_tablet_manager.cc:1434] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.047s user 0.005s sys 0.000s
I20260926 19:58:54.722955 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 240) took 1457 ms. Trace:
I20260926 19:58:54.723274 13403 rpcz_store.cc:276] 0926 19:58:53.265128 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.265269 (+ 141us) service_pool.cc:224] Handling call
0926 19:58:54.281743 (+1016474us) ts_tablet_manager.cc:1151] Deleting tablet 66c8916d88dd48998eab514bd3c538b9
0926 19:58:54.281785 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.589351 (+307566us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.625244 (+ 35893us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.689070 (+ 63826us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b5800000f98. stack: 00007ffa5c912323 00007ffa5c912220 00007ffa5c4b6574 0000000000516964 00000000005168f9 00007ffa5c7c88a1 00007ffa5c90cb51 00007ffa5c90d182 00007ffa658802c9 00007ffa64a91918 00007ffa64b58d82 00007ffa64b89960 00007ffa64b6fe6d 00007ffa64b6fe19 00007ffa64b6fda9 00007ffa64b6fd71
0926 19:58:54.722884 (+ 33814us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43672,"spinlock_wait_cycles":15651712,"tablet-delete.queue_time_us":1015791}
I20260926 19:58:54.724890 13403 tablet_replica.cc:333] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:54.717787 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bdb479c5384d0891db66943ef263b2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:54.726774 12696 leader_election.cc:343] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.729772 13092 tablet_bootstrap.cc:654] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.721098 13715 raft_consensus.cc:515] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.734228 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 16897e7489e9445ba5389a4f3507a739 (table default.Table_d7b3b0491c1a417ea7d465c4662e7dee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]) successfully deleted
I20260926 19:58:54.732613 13012 tablet_bootstrap.cc:492] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:54.735217 13012 ts_tablet_manager.cc:1403] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.079s user 0.009s sys 0.001s
I20260926 19:58:54.737877 13012 raft_consensus.cc:359] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.741194 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d48ee26c0aaa4abc8e08a1a2dadb028d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:54.742741 13012 raft_consensus.cc:385] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.743083 13012 raft_consensus.cc:740] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.746680 13013 tablet_bootstrap.cc:492] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:54.746978 13715 leader_election.cc:290] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:54.749778 13012 consensus_queue.cc:260] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.751237 13012 ts_tablet_manager.cc:1434] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260926 19:58:54.753588 12858 leader_election.cc:304] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:54.757562 13569 ts_tablet_manager.cc:1929] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:54.757918 13569 log.cc:1199] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/16df0828847d4e7db5702d84082fa402
I20260926 19:58:54.758662 13569 ts_tablet_manager.cc:1950] T 16df0828847d4e7db5702d84082fa402 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:54.760491 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 240) took 1914 ms. Trace:
I20260926 19:58:54.760795 13569 rpcz_store.cc:276] 0926 19:58:52.845866 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.845963 (+ 97us) service_pool.cc:224] Handling call
0926 19:58:54.614787 (+1768824us) ts_tablet_manager.cc:1151] Deleting tablet 16df0828847d4e7db5702d84082fa402
0926 19:58:54.614828 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.691973 (+ 77145us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.757424 (+ 65451us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.760436 (+ 3012us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1768321}
I20260926 19:58:54.768680 13013 tablet_bootstrap.cc:654] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.772334 13203 ts_tablet_manager.cc:1929] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:54.772770 13203 log.cc:1199] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/fb593ce2cd4147febebfe442de9fdd28
I20260926 19:58:54.774755 13012 tablet_bootstrap.cc:492] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:54.774804 12389 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 0df471a7b6554d4fb5007adb63babe43
I20260926 19:58:54.778923 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 0df471a7b6554d4fb5007adb63babe43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:54 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:54.779747 13203 ts_tablet_manager.cc:1950] T fb593ce2cd4147febebfe442de9fdd28 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:54.780985 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet 0df471a7b6554d4fb5007adb63babe43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:54 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:54.793283 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 254) took 1368 ms. Trace:
I20260926 19:58:54.793468 13403 raft_consensus.cc:2243] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:54.794461 13403 raft_consensus.cc:2272] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.796836 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 16df0828847d4e7db5702d84082fa402 (table default.Table_29ce86fb5af04ec4b8411a4f3135b79d [id=74eae66d927e413a9457c4c89cff78ce]) successfully deleted
I20260926 19:58:54.798372 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 241) took 1808 ms. Trace:
I20260926 19:58:54.801172 13569 rpcz_store.cc:276] 0926 19:58:52.989626 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.989984 (+ 358us) service_pool.cc:224] Handling call
0926 19:58:54.798071 (+1808087us) ts_tablet_manager.cc:1151] Deleting tablet 56e337767be84507acdecfb189705bdd
0926 19:58:54.798121 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.798304 (+ 183us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1807538}
I20260926 19:58:54.793653 13203 rpcz_store.cc:276] 0926 19:58:53.424820 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.425132 (+ 312us) service_pool.cc:224] Handling call
0926 19:58:54.628328 (+1203196us) ts_tablet_manager.cc:1151] Deleting tablet fb593ce2cd4147febebfe442de9fdd28
0926 19:58:54.628369 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.655873 (+ 27504us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.772192 (+116319us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.793217 (+ 21025us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1202670}
I20260926 19:58:54.799559 13012 tablet_bootstrap.cc:654] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.804299 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet fb593ce2cd4147febebfe442de9fdd28 (table default.Table_693c5b4b4dc446378398965b02ddeea5 [id=d088bdbe9cf34c41adfc15548ca5521a]) successfully deleted
W20260926 19:58:54.805316 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 56e337767be84507acdecfb189705bdd because the tablet was not found. No further retry: Not found: Tablet not found: 56e337767be84507acdecfb189705bdd
I20260926 19:58:54.807286 13569 tablet_replica.cc:333] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
W20260926 19:58:54.810312 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 6c05c67d89814ca6bc906f4b683523b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c05c67d89814ca6bc906f4b683523b0 already in progress: creating tablet
I20260926 19:58:54.810496 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 66c8916d88dd48998eab514bd3c538b9 (table default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]) successfully deleted
I20260926 19:58:54.811393 13203 tablet_replica.cc:333] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:54.812093 13203 raft_consensus.cc:2243] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:54.812523 13203 raft_consensus.cc:2272] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.811381 13569 raft_consensus.cc:2243] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:54.812619 12394 master_service.cc:960] ReplaceTablet: received request to replace tablet f2663906e11549e3aa9ae78d88e645ea from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:54.815222 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_4144e2041c67465ea697a20aa6b1800b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:54.818131 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f42b2a59275343c796ebf6f3e3e96154"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:54.819626 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f42b2a59275343c796ebf6f3e3e96154 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:54.821754 13715 raft_consensus.cc:493] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:54.822209 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_94d1edd7294d4dc38229784cab1912ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:54.822250 13715 raft_consensus.cc:515] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:54.823728 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94d1edd7294d4dc38229784cab1912ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:54.824359 13715 leader_election.cc:290] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:54.825327 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b294fbdc488a45a3b54ff6e45272ef72" } new_table_name: "default.Table_82df428358de4fc18b05ae70c217854b" modify_external_catalogs: true
I20260926 19:58:54.821372 13569 raft_consensus.cc:2272] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.827270 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bdb479c5384d0891db66943ef263b2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:54.827829 12594 raft_consensus.cc:3060] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:54.828476 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:54.829169 12589 raft_consensus.cc:2393] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:54.832727 13569 ts_tablet_manager.cc:1916] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.833271 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:54.835180 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 8ff071a16f1244cebc8c93e5508280a3 (table default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7 [id=e452740c55c345a9ab7445009c7a71cb]). Expected version 2 got 0
I20260926 19:58:54.836139 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
W20260926 19:58:54.836233 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4144e2041c67465ea697a20aa6b1800b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:54.836426 12594 raft_consensus.cc:2468] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:54.838936 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_86ffbd3fac614b7db1a61cba6ab82328"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:54.839846 13203 ts_tablet_manager.cc:1916] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:54.840696 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86ffbd3fac614b7db1a61cba6ab82328 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:54.841361 13012 tablet_bootstrap.cc:492] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:54.841797 13012 ts_tablet_manager.cc:1403] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.067s user 0.009s sys 0.002s
I20260926 19:58:54.842069 13013 tablet_bootstrap.cc:492] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:54.842672 13013 ts_tablet_manager.cc:1403] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.096s user 0.011s sys 0.000s
I20260926 19:58:54.844712 13012 raft_consensus.cc:359] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.844871 13013 raft_consensus.cc:359] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.849869 13012 raft_consensus.cc:385] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.850639 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:54.851095 13012 raft_consensus.cc:740] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.851732 13012 consensus_queue.cc:260] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.851989 12392 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_cc7b2242e1c44b9cabad45093975a899 [id=d9c3ce1d8dd440c2862eb5fa4c58bf72] alter complete (version 1)
I20260926 19:58:54.852855 12396 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_bfdeb281a21842bc9cf53aafa87f80cd [id=dd5c0a6b728948278a5dd48f8ec0bf6f] alter complete (version 3)
W20260926 19:58:54.856686 12396 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 62) took 902 ms (client timeout 1000 ms). Trace:
I20260926 19:58:54.845467 13013 raft_consensus.cc:385] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.859840 13013 raft_consensus.cc:740] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.860453 13013 consensus_queue.cc:260] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.862136 13012 ts_tablet_manager.cc:1434] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.020s user 0.004s sys 0.001s
I20260926 19:58:54.869630 13012 tablet_bootstrap.cc:492] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:54.872109 13013 ts_tablet_manager.cc:1434] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260926 19:58:54.872952 13013 tablet_bootstrap.cc:492] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
W20260926 19:58:54.857105 12396 rpcz_store.cc:269] 0926 19:58:53.954245 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.954346 (+ 101us) service_pool.cc:224] Handling call
0926 19:58:53.961201 (+ 6855us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:54.035043 (+ 73842us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b54000303a8. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36b01e1a1 00007fb36b0a85bf 00007fb36b0a9d21 00007fb36cb0fef1 00007fb36cb1074f 00007fb36c9d1627 00007fb36c9cfa00 00007fb36cad78de 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71
0926 19:58:54.856575 (+821532us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:54.435918 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:54.436280 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:54.436303 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:58:54.436316 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:54.436574 (+ 258us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:54.436610 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:54.436631 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:54.436773 (+ 142us) tablet.cc:701] Row locks acquired
0926 19:58:54.436786 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:54.436882 (+ 96us) write_op.cc:270] Start()
0926 19:58:54.436955 (+ 73us) write_op.cc:276] Timestamp: P: 1790452734436830 usec, L: 0
0926 19:58:54.436971 (+ 16us) op_driver.cc:348] REPLICATION: starting
0926 19:58:54.437410 (+ 439us) log.cc:844] Serialized 356 byte log entry
0926 19:58:54.469024 (+ 31614us) op_driver.cc:464] REPLICATION: finished
0926 19:58:54.470702 (+ 1678us) write_op.cc:301] APPLY: starting
0926 19:58:54.470768 (+ 66us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:54.470846 (+ 78us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:54.470947 (+ 101us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:54.471081 (+ 134us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:54.471199 (+ 118us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:54.471241 (+ 42us) write_op.cc:312] APPLY: finished
0926 19:58:54.471387 (+ 146us) log.cc:844] Serialized 23 byte log entry
0926 19:58:54.471525 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:54.471576 (+ 51us) write_op.cc:454] Released schema lock
0926 19:58:54.471604 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:54.471816 (+ 212us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:54.471839 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":108056320,"child_traces":[["op",{"apply.queue_time_us":1521,"apply.run_cpu_time_us":1083,"apply.run_wall_time_us":1084,"num_ops":1,"prepare.queue_time_us":400349,"prepare.run_cpu_time_us":1758,"prepare.run_wall_time_us":7695,"raft.queue_time_us":207,"raft.run_cpu_time_us":629,"raft.run_wall_time_us":628,"replication_time_us":31956}]]}
I20260926 19:58:54.889142 12396 master_service.cc:960] ReplaceTablet: received request to replace tablet d734112eca1249559d0ff3d123e82042 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:54.891487 13012 tablet_bootstrap.cc:654] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.903725 13013 tablet_bootstrap.cc:654] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:54.904754 13569 ts_tablet_manager.cc:1929] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:54.905660 13569 log.cc:1199] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/77b319b685344a848462b78617eb6c22
I20260926 19:58:54.906726 13807 raft_consensus.cc:493] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:54.907215 13807 raft_consensus.cc:515] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:54.907987 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 441b881f3660495cba9b94a8c7464e99 (table default.Table_42e2147189dd490180e130039faf6cdc [id=fedf8b09ae88433aa308e7fbf4c4fac6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 441b881f3660495cba9b94a8c7464e99 has no consensus state
I20260926 19:58:54.909241 13807 leader_election.cc:290] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:54.911026 13569 ts_tablet_manager.cc:1950] T 77b319b685344a848462b78617eb6c22 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:54.920704 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 242) took 1929 ms. Trace:
I20260926 19:58:54.924566 13569 rpcz_store.cc:276] 0926 19:58:52.990979 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:52.991475 (+ 496us) service_pool.cc:224] Handling call
0926 19:58:54.807103 (+1815628us) ts_tablet_manager.cc:1151] Deleting tablet 77b319b685344a848462b78617eb6c22
0926 19:58:54.807152 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.870780 (+ 63628us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.904636 (+ 33856us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:54.920648 (+ 16012us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1814404}
I20260926 19:58:54.927517 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 77b319b685344a848462b78617eb6c22 (table default.Table_a958288944144fbd9cc934add059cdb0 [id=1850b06b63d24da0b91ad1a54eae3dd9]) successfully deleted
I20260926 19:58:54.926606 13012 tablet_bootstrap.cc:492] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:54.928584 13012 ts_tablet_manager.cc:1403] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.059s user 0.008s sys 0.002s
I20260926 19:58:54.931865 13012 raft_consensus.cc:359] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.935467 13203 ts_tablet_manager.cc:1929] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:54.935822 13203 log.cc:1199] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/2ad76fc959e24545bdfad19f988fce93
I20260926 19:58:54.934149 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 245) took 1909 ms. Trace:
I20260926 19:58:54.936439 13203 ts_tablet_manager.cc:1950] T 2ad76fc959e24545bdfad19f988fce93 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:54.935851 13012 raft_consensus.cc:385] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.937748 13012 raft_consensus.cc:740] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:54.938268 13825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.253:37913 every 8 connection(s)
I20260926 19:58:54.936182 13569 rpcz_store.cc:276] 0926 19:58:53.025053 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.025165 (+ 112us) service_pool.cc:224] Handling call
0926 19:58:54.933807 (+1908642us) ts_tablet_manager.cc:1151] Deleting tablet 5930402db43740b4b291c6c238ecf52c
0926 19:58:54.933849 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.934089 (+ 240us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1908066}
I20260926 19:58:54.938860 13130 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.253:37913
I20260926 19:58:54.939417 12391 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet f1fb09495d1b46e894f129d342e9986e
W20260926 19:58:54.939486 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 5930402db43740b4b291c6c238ecf52c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5930402db43740b4b291c6c238ecf52c already in progress: creating tablet
I20260926 19:58:54.941282 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 246) took 1890 ms. Trace:
I20260926 19:58:54.944082 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2d4d45fdeb4802ac261b6c8de91c3c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:54.944736 12594 raft_consensus.cc:2468] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:54.945596 13569 rpcz_store.cc:276] 0926 19:58:53.050965 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.051076 (+ 111us) service_pool.cc:224] Handling call
0926 19:58:54.940942 (+1889866us) ts_tablet_manager.cc:1151] Deleting tablet 8c7d0435f3564cb38b6cbca713009c2e
0926 19:58:54.940981 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:54.941228 (+ 247us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1889351}
W20260926 19:58:54.946893 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 8c7d0435f3564cb38b6cbca713009c2e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c7d0435f3564cb38b6cbca713009c2e already in progress: creating tablet
I20260926 19:58:54.947130 12695 leader_election.cc:304] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:54.947916 13807 raft_consensus.cc:2804] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:54.948330 13807 raft_consensus.cc:697] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:54.941843 13012 consensus_queue.cc:260] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.949066 13807 consensus_queue.cc:237] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.951890 13130 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
I20260926 19:58:54.956168 13569 tablet_replica.cc:333] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:54.957973 13569 raft_consensus.cc:2243] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:54.958379 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 2ad76fc959e24545bdfad19f988fce93 (table default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189]) successfully deleted
I20260926 19:58:54.960544 13013 tablet_bootstrap.cc:492] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:54.960949 13013 ts_tablet_manager.cc:1403] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.088s user 0.008s sys 0.000s
I20260926 19:58:54.962208 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 13130
I20260926 19:58:54.962131 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.962962 12920 raft_consensus.cc:2393] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ae87963ed5e4d92a896bf4a73af4da9 in current term 1: Already voted for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in this term.
I20260926 19:58:54.963894 13013 raft_consensus.cc:359] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:54.964564 13013 raft_consensus.cc:385] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:54.964816 13013 raft_consensus.cc:740] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:58:54.968535 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 13bb4e7eeef145fc85a5c412add99ce7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13bb4e7eeef145fc85a5c412add99ce7 already in progress: creating tablet
I20260926 19:58:54.970741 13203 tablet_replica.cc:333] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:54.972316 13569 raft_consensus.cc:2272] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.973376 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bdb479c5384d0891db66943ef263b2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:54.974462 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet 0df471a7b6554d4fb5007adb63babe43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:54 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:54.958807 13012 ts_tablet_manager.cc:1434] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.028s user 0.005s sys 0.001s
I20260926 19:58:54.975610 13012 tablet_bootstrap.cc:492] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:54.981786 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2d4d45fdeb4802ac261b6c8de91c3c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:54.983129 13569 ts_tablet_manager.cc:1916] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:54.971493 13203 raft_consensus.cc:2243] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:54.991057 13203 raft_consensus.cc:2272] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:54.992481 12696 leader_election.cc:304] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:58:54.993669 12696 leader_election.cc:343] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:54.994004 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 46976.4us
W20260926 19:58:54.994455 12696 leader_election.cc:343] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:54.994849 12695 leader_election.cc:304] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:54.995450 13715 raft_consensus.cc:3060] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:54.995890 13599 raft_consensus.cc:2804] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:54.996212 13599 raft_consensus.cc:493] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:54.996500 13599 raft_consensus.cc:3060] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:54.997303 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_d7b3b0491c1a417ea7d465c4662e7dee" } new_table_name: "default.Table_646a921cdd5c47d988e91966a807b79b" modify_external_catalogs: true
I20260926 19:58:55.003602 13599 raft_consensus.cc:515] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.006335 13599 leader_election.cc:290] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:55.012032 13403 ts_tablet_manager.cc:1916] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.000880 13013 consensus_queue.cc:260] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.014701 13013 ts_tablet_manager.cc:1434] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.053s user 0.007s sys 0.000s
I20260926 19:58:55.015552 13013 tablet_bootstrap.cc:492] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:54.993384 13203 ts_tablet_manager.cc:1916] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.022698 13012 tablet_bootstrap.cc:654] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.026882 13549 raft_consensus.cc:493] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.027451 13549 raft_consensus.cc:515] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.030681 13549 leader_election.cc:290] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:54.807271 13785 raft_consensus.cc:2749] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:58:55.038091 12395 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_159e9c03205e4e989b5e0c296bca3da0 [id=dfc3435de1e7453794433737b7c342fa] alter complete (version 1)
I20260926 19:58:55.043220 13569 ts_tablet_manager.cc:1929] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.057699 13569 log.cc:1199] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/16897e7489e9445ba5389a4f3507a739
I20260926 19:58:55.057862 13203 ts_tablet_manager.cc:1929] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.058211 13203 log.cc:1199] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/cd86f01d82d041f4852f83f2547e697f
I20260926 19:58:55.058636 13569 ts_tablet_manager.cc:1950] T 16897e7489e9445ba5389a4f3507a739 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:55.058827 13203 ts_tablet_manager.cc:1950] T cd86f01d82d041f4852f83f2547e697f P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:55.061159 13830 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:58:55.061300 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2d4d45fdeb4802ac261b6c8de91c3c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
W20260926 19:58:55.066756 12696 leader_election.cc:343] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.068666 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89870d2c5b0342c4bdf66474fa8d38fe" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:55.069024 13827 log.cc:927] Time spent T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
W20260926 19:58:55.071517 12519 leader_election.cc:343] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.072266 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89870d2c5b0342c4bdf66474fa8d38fe" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:55.073822 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 78172.1us
I20260926 19:58:55.060360 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet 16897e7489e9445ba5389a4f3507a739 with delete_type TABLET_DATA_DELETED (replaced by tablet 986a28107bd54199bf2cda96978150c3) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:55.049961 13748 tablet.cc:1751] T 8ff071a16f1244cebc8c93e5508280a3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260926 19:58:55.080178 13715 raft_consensus.cc:2749] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:55.087774 12521 leader_election.cc:343] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.088297 12521 leader_election.cc:304] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:55.088811 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet cd86f01d82d041f4852f83f2547e697f (table default.Table_87cff927dd5d4f0d9d947b7a5629728c [id=6a77f8956ec6491d915b40237d54a548]) successfully deleted
I20260926 19:58:55.089812 13203 tablet_replica.cc:333] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:55.093647 13203 raft_consensus.cc:2243] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:55.100927 13203 raft_consensus.cc:2272] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:55.105665 13549 raft_consensus.cc:2749] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:55.107100 13012 tablet_bootstrap.cc:492] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:55.107491 13012 ts_tablet_manager.cc:1403] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.132s user 0.003s sys 0.010s
I20260926 19:58:55.108337 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 247) took 1923 ms. Trace:
I20260926 19:58:55.108700 13569 rpcz_store.cc:276] 0926 19:58:53.185205 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.185764 (+ 559us) service_pool.cc:224] Handling call
0926 19:58:54.956004 (+1770240us) ts_tablet_manager.cc:1151] Deleting tablet 16897e7489e9445ba5389a4f3507a739
0926 19:58:54.956050 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.028891 (+ 72841us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.043087 (+ 14196us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.108282 (+ 65195us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1724231}
I20260926 19:58:55.109534 13803 raft_consensus.cc:493] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.109859 13012 raft_consensus.cc:359] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.110486 13012 raft_consensus.cc:385] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.110097 13803 raft_consensus.cc:515] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.110743 13012 raft_consensus.cc:740] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.110805 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 16897e7489e9445ba5389a4f3507a739 (table default.Table_d7b3b0491c1a417ea7d465c4662e7dee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]) successfully deleted
I20260926 19:58:55.112483 13803 leader_election.cc:290] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:55.112805 13569 tablet_replica.cc:333] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:55.105495 13203 ts_tablet_manager.cc:1916] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.114065 13569 raft_consensus.cc:2243] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
W20260926 19:58:55.115545 12858 leader_election.cc:343] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: c5b734f0d33342d3be8b1827bcd5dd14
I20260926 19:58:55.115996 13830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Bootstrap starting.
I20260926 19:58:55.117071 13569 raft_consensus.cc:2272] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:55.118768 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 44351us
I20260926 19:58:55.113821 13012 consensus_queue.cc:260] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.132071 13012 ts_tablet_manager.cc:1434] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260926 19:58:55.133188 13012 tablet_bootstrap.cc:492] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:55.133488 13569 ts_tablet_manager.cc:1916] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.139496 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2d4d45fdeb4802ac261b6c8de91c3c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:55.140087 12594 raft_consensus.cc:3060] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:55.143304 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5b734f0d33342d3be8b1827bcd5dd14" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:55.144006 12759 raft_consensus.cc:2393] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate f6e9d6439fd54b28925a68a6145d3dee in this term.
I20260926 19:58:55.145263 12857 leader_election.cc:304] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:55.147681 12392 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_3246f278dd5e48c9bf8e940cc4ee0f7c [id=faf55c4cd53c42a9b894d47f5064d220] alter complete (version 1)
I20260926 19:58:55.148754 12594 raft_consensus.cc:2468] T 9b2d4d45fdeb4802ac261b6c8de91c3c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:55.151645 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet 16df0828847d4e7db5702d84082fa402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:42 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:55.151698 12392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 53) took 1155 ms (client timeout 1000 ms). Trace:
I20260926 19:58:55.153127 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:55.146387 13803 raft_consensus.cc:3060] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:55.151989 12392 rpcz_store.cc:269] 0926 19:58:53.996689 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.040545 (+ 43856us) service_pool.cc:224] Handling call
0926 19:58:55.151612 (+1111067us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:54.453080 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:54.453426 (+ 346us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:54.453446 (+ 20us) write_op.cc:435] Acquired schema lock
0926 19:58:54.453458 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:54.453706 (+ 248us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:54.453747 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:54.453769 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:54.453909 (+ 140us) tablet.cc:701] Row locks acquired
0926 19:58:54.453923 (+ 14us) write_op.cc:260] PREPARE: finished
0926 19:58:54.454014 (+ 91us) write_op.cc:270] Start()
0926 19:58:54.454079 (+ 65us) write_op.cc:276] Timestamp: P: 1790452734453964 usec, L: 0
0926 19:58:54.454097 (+ 18us) op_driver.cc:348] REPLICATION: starting
0926 19:58:54.465710 (+ 11613us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5400030d20. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36abdba50 00007fb36ac65302 00007fb36ac5a727 00007fb36ac5a564 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11
0926 19:58:54.466249 (+ 539us) log.cc:844] Serialized 356 byte log entry
0926 19:58:54.495568 (+ 29319us) op_driver.cc:464] REPLICATION: finished
0926 19:58:54.497677 (+ 2109us) write_op.cc:301] APPLY: starting
0926 19:58:54.497726 (+ 49us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:54.497817 (+ 91us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:54.497836 (+ 19us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:54.497988 (+ 152us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:54.498128 (+ 140us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:54.498188 (+ 60us) write_op.cc:312] APPLY: finished
0926 19:58:54.498372 (+ 184us) log.cc:844] Serialized 23 byte log entry
0926 19:58:54.499235 (+ 863us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:54.499295 (+ 60us) write_op.cc:454] Released schema lock
0926 19:58:54.499325 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:54.499559 (+ 234us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:54.499583 (+ 24us) write_op.cc:454] Released schema lock
Related trace 'op':
0926 19:58:54.868674 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:54.869086 (+ 412us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:54.869115 (+ 29us) write_op.cc:435] Acquired schema lock
0926 19:58:54.869128 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:54.869372 (+ 244us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:54.869404 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:54.869432 (+ 28us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:54.869629 (+ 197us) tablet.cc:701] Row locks acquired
0926 19:58:54.869659 (+ 30us) write_op.cc:260] PREPARE: finished
0926 19:58:54.869791 (+ 132us) write_op.cc:270] Start()
0926 19:58:54.869895 (+ 104us) write_op.cc:276] Timestamp: P: 1790452734869707 usec, L: 0
0926 19:58:54.869916 (+ 21us) op_driver.cc:348] REPLICATION: starting
0926 19:58:54.870490 (+ 574us) log.cc:844] Serialized 356 byte log entry
0926 19:58:55.117869 (+247379us) op_driver.cc:464] REPLICATION: finished
0926 19:58:55.145834 (+ 27965us) write_op.cc:301] APPLY: starting
0926 19:58:55.145888 (+ 54us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:55.145981 (+ 93us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:55.146079 (+ 98us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:55.146290 (+ 211us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:55.146434 (+ 144us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:55.146476 (+ 42us) write_op.cc:312] APPLY: finished
0926 19:58:55.146686 (+ 210us) log.cc:844] Serialized 23 byte log entry
0926 19:58:55.147270 (+ 584us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:55.147329 (+ 59us) write_op.cc:454] Released schema lock
0926 19:58:55.147360 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":1946,"apply.run_cpu_time_us":1167,"apply.run_wall_time_us":1855,"num_ops":1,"prepare.queue_time_us":224766,"prepare.run_cpu_time_us":2848,"prepare.run_wall_time_us":42769,"replication_time_us":41376,"spinlock_wait_cycles":24411904}],["op",{"apply.queue_time_us":27768,"num_ops":1,"prepare.queue_time_us":9380,"prepare.run_cpu_time_us":2090,"prepare.run_wall_time_us":51678,"replication_time_us":247852}]]}
I20260926 19:58:55.162644 12695 leader_election.cc:304] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:55.163614 13840 raft_consensus.cc:2804] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:55.164391 13092 tablet_bootstrap.cc:492] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:55.164791 13092 ts_tablet_manager.cc:1403] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.531s user 0.010s sys 0.014s
I20260926 19:58:55.167121 13092 raft_consensus.cc:359] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.167745 13092 raft_consensus.cc:385] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.167997 13092 raft_consensus.cc:740] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.168257 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5bf4644212d544c696bffa35c366b0f7" } modify_external_catalogs: true
I20260926 19:58:55.168838 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5bf4644212d544c696bffa35c366b0f7" } modify_external_catalogs: true
I20260926 19:58:55.168699 13092 consensus_queue.cc:260] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.169605 13803 raft_consensus.cc:2749] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:55.170897 13840 raft_consensus.cc:697] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
W20260926 19:58:55.173147 13717 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s user 0.037s sys 0.037s
W20260926 19:58:55.173437 13717 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s user 0.037s sys 0.037s
I20260926 19:58:55.178026 13403 ts_tablet_manager.cc:1929] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:55.178377 13403 log.cc:1199] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/16df0828847d4e7db5702d84082fa402
I20260926 19:58:55.179064 13403 ts_tablet_manager.cc:1950] T 16df0828847d4e7db5702d84082fa402 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:55.177265 12391 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 8ff071a16f1244cebc8c93e5508280a3 (table default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7 [id=e452740c55c345a9ab7445009c7a71cb]). Expected version 2 got 0
I20260926 19:58:55.183784 12391 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_aa7b619547ec4df09f564d512fb394c7" } modify_external_catalogs: true
I20260926 19:58:55.183929 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:55.184295 12391 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_aa7b619547ec4df09f564d512fb394c7" } modify_external_catalogs: true
I20260926 19:58:55.200260 13092 ts_tablet_manager.cc:1434] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260926 19:58:55.201161 13092 tablet_bootstrap.cc:492] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:55.202339 13569 ts_tablet_manager.cc:1929] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:55.219753 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 242) took 1953 ms. Trace:
I20260926 19:58:55.219973 13569 log.cc:1199] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/fb593ce2cd4147febebfe442de9fdd28
I20260926 19:58:55.172120 13840 consensus_queue.cc:237] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.220057 13403 rpcz_store.cc:276] 0926 19:58:53.266702 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.266804 (+ 102us) service_pool.cc:224] Handling call
0926 19:58:54.724734 (+1457930us) ts_tablet_manager.cc:1151] Deleting tablet 16df0828847d4e7db5702d84082fa402
0926 19:58:54.724775 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.058214 (+333439us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.177909 (+119695us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.219692 (+ 41783us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1048,"tablet-delete.queue_time_us":1457367}
I20260926 19:58:55.220723 13569 ts_tablet_manager.cc:1950] T fb593ce2cd4147febebfe442de9fdd28 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:55.221309 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 16df0828847d4e7db5702d84082fa402 (table default.Table_29ce86fb5af04ec4b8411a4f3135b79d [id=74eae66d927e413a9457c4c89cff78ce]) successfully deleted
I20260926 19:58:55.222224 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 241) took 1955 ms. Trace:
I20260926 19:58:55.222414 13092 tablet_bootstrap.cc:654] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.222723 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 249) took 1827 ms. Trace:
I20260926 19:58:55.222522 13403 rpcz_store.cc:276] 0926 19:58:53.266387 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.267063 (+ 676us) service_pool.cc:224] Handling call
0926 19:58:55.221976 (+1954913us) ts_tablet_manager.cc:1151] Deleting tablet 56e337767be84507acdecfb189705bdd
0926 19:58:55.222014 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.222171 (+ 157us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1954333}
I20260926 19:58:55.223032 13569 rpcz_store.cc:276] 0926 19:58:53.394882 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.396735 (+ 1853us) service_pool.cc:224] Handling call
0926 19:58:55.112637 (+1715902us) ts_tablet_manager.cc:1151] Deleting tablet fb593ce2cd4147febebfe442de9fdd28
0926 19:58:55.112686 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.181178 (+ 68492us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.202221 (+ 21043us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.222661 (+ 20440us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1652222}
W20260926 19:58:55.224126 12382 catalog_manager.cc:5030] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 56e337767be84507acdecfb189705bdd because the tablet was not found. No further retry: Not found: Tablet not found: 56e337767be84507acdecfb189705bdd
I20260926 19:58:55.224164 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet fb593ce2cd4147febebfe442de9fdd28 (table default.Table_693c5b4b4dc446378398965b02ddeea5 [id=d088bdbe9cf34c41adfc15548ca5521a]) successfully deleted
I20260926 19:58:55.225158 13569 tablet_replica.cc:333] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:55.225922 13569 raft_consensus.cc:2243] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:55.226429 13569 raft_consensus.cc:2272] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:55.228029 13569 ts_tablet_manager.cc:1916] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.230070 13203 ts_tablet_manager.cc:1929] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.230436 13203 log.cc:1199] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/06c5ef9e1bca427fb889cbb17f77f999
I20260926 19:58:55.231050 13203 ts_tablet_manager.cc:1950] T 06c5ef9e1bca427fb889cbb17f77f999 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:55.249295 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 276) took 1028 ms. Trace:
I20260926 19:58:55.249655 13203 rpcz_store.cc:276] 0926 19:58:54.220511 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.220623 (+ 112us) service_pool.cc:224] Handling call
0926 19:58:55.089666 (+869043us) ts_tablet_manager.cc:1151] Deleting tablet 06c5ef9e1bca427fb889cbb17f77f999
0926 19:58:55.089712 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.155883 (+ 66171us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.229951 (+ 74068us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.249227 (+ 19276us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":868528}
I20260926 19:58:55.250860 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 06c5ef9e1bca427fb889cbb17f77f999 (table default.Table_06d7f301c3cb4d0e8d374df103fab2f2 [id=ab940466309d4545b5a3f15ecc020742]) successfully deleted
I20260926 19:58:55.253921 13203 tablet_replica.cc:333] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:55.255071 13092 tablet_bootstrap.cc:492] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:55.255462 13092 ts_tablet_manager.cc:1403] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.000s
I20260926 19:58:55.256722 13203 raft_consensus.cc:2243] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:55.257375 13203 pending_rounds.cc:70] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Trying to abort 1 pending ops.
I20260926 19:58:55.257611 13203 pending_rounds.cc:77] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694400676843520 op_type: NO_OP noop_request { }
I20260926 19:58:55.257931 13203 raft_consensus.cc:2889] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:55.257798 13092 raft_consensus.cc:359] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.258368 13092 raft_consensus.cc:385] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.258615 13092 raft_consensus.cc:740] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.260923 13092 consensus_queue.cc:260] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.264257 13012 tablet_bootstrap.cc:654] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.266187 13569 ts_tablet_manager.cc:1929] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.270929 13092 ts_tablet_manager.cc:1434] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.015s user 0.003s sys 0.003s
I20260926 19:58:55.271317 13569 log.cc:1199] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/66c8916d88dd48998eab514bd3c538b9
I20260926 19:58:55.272025 13569 ts_tablet_manager.cc:1950] T 66c8916d88dd48998eab514bd3c538b9 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:55.258244 13203 raft_consensus.cc:2272] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:55.273777 13092 tablet_bootstrap.cc:492] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:55.274096 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 254) took 1408 ms. Trace:
I20260926 19:58:55.274444 13569 rpcz_store.cc:276] 0926 19:58:53.866042 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.866163 (+ 121us) service_pool.cc:224] Handling call
0926 19:58:55.225019 (+1358856us) ts_tablet_manager.cc:1151] Deleting tablet 66c8916d88dd48998eab514bd3c538b9
0926 19:58:55.225057 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.239054 (+ 13997us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.266075 (+ 27021us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.274042 (+ 7967us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1357567}
I20260926 19:58:55.276849 13013 tablet_bootstrap.cc:654] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.262833 13830 log.cc:826] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Log is configured to *not* fsync() on all Append() calls
I20260926 19:58:55.277767 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 66c8916d88dd48998eab514bd3c538b9 (table default.Table_21e4b470924b49518cf4ca4b8e336b36 [id=bcfdb47fb4a643f39106a26537d6f4d9]) successfully deleted
I20260926 19:58:55.278898 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 266) took 1231 ms. Trace:
I20260926 19:58:55.279322 13569 rpcz_store.cc:276] 0926 19:58:54.047951 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.048175 (+ 224us) service_pool.cc:224] Handling call
0926 19:58:55.278547 (+1230372us) ts_tablet_manager.cc:1151] Deleting tablet 6c05c67d89814ca6bc906f4b683523b0
0926 19:58:55.278588 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.278837 (+ 249us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1229651}
I20260926 19:58:55.285214 13203 ts_tablet_manager.cc:1916] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:55.286376 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 6c05c67d89814ca6bc906f4b683523b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c05c67d89814ca6bc906f4b683523b0 already in progress: creating tablet
I20260926 19:58:55.287562 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 269) took 1169 ms. Trace:
I20260926 19:58:55.287990 13569 rpcz_store.cc:276] 0926 19:58:54.118393 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.118598 (+ 205us) service_pool.cc:224] Handling call
0926 19:58:55.287190 (+1168592us) ts_tablet_manager.cc:1151] Deleting tablet 13bb4e7eeef145fc85a5c412add99ce7
0926 19:58:55.287244 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.287505 (+ 261us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1167519}
W20260926 19:58:55.289399 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 13bb4e7eeef145fc85a5c412add99ce7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13bb4e7eeef145fc85a5c412add99ce7 already in progress: creating tablet
I20260926 19:58:55.290582 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 270) took 1170 ms. Trace:
I20260926 19:58:55.290905 13569 rpcz_store.cc:276] 0926 19:58:54.120233 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.120328 (+ 95us) service_pool.cc:224] Handling call
0926 19:58:55.290283 (+1169955us) ts_tablet_manager.cc:1151] Deleting tablet 2ad76fc959e24545bdfad19f988fce93
0926 19:58:55.290319 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.290531 (+ 212us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1169437}
W20260926 19:58:55.292098 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 2ad76fc959e24545bdfad19f988fce93 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ad76fc959e24545bdfad19f988fce93 already in progress: creating tablet
W20260926 19:58:55.293728 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 0df471a7b6554d4fb5007adb63babe43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0df471a7b6554d4fb5007adb63babe43 already in progress: creating tablet
W20260926 19:58:55.295193 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 16df0828847d4e7db5702d84082fa402 because the tablet was not found. No further retry: Not found: Tablet not found: 16df0828847d4e7db5702d84082fa402
I20260926 19:58:55.295981 13569 tablet_replica.cc:333] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:55.296823 13569 raft_consensus.cc:2243] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:55.301925 13569 raft_consensus.cc:2272] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:55.306041 13569 ts_tablet_manager.cc:1916] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.307889 13092 tablet_bootstrap.cc:654] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.308630 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 237) took 2070 ms. Trace:
I20260926 19:58:55.322240 12387 master_service.cc:960] ReplaceTablet: received request to replace tablet de85e3e17bf340f8895aff3f5421c85c from {username='slave'} at 127.0.0.1:45734
W20260926 19:58:55.328364 12390 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3df392d277a447aabf4320a7d9a43880 consensus state has no leader
I20260926 19:58:55.331101 13717 raft_consensus.cc:493] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.332036 13092 tablet_bootstrap.cc:492] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:55.332427 13092 ts_tablet_manager.cc:1403] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.059s user 0.005s sys 0.004s
I20260926 19:58:55.331692 13717 raft_consensus.cc:515] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.339178 13717 leader_election.cc:290] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:55.341686 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bdb479c5384d0891db66943ef263b2" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:55.342257 12807 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
I20260926 19:58:55.342427 12594 raft_consensus.cc:2393] T 46bdb479c5384d0891db66943ef263b2 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:58:55.345893 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46bdb479c5384d0891db66943ef263b2" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:55.347442 12857 leader_election.cc:304] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:55.348515 13717 raft_consensus.cc:3060] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:55.308975 13403 rpcz_store.cc:276] 0926 19:58:53.238606 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.249565 (+ 10959us) service_pool.cc:224] Handling call
0926 19:58:55.308271 (+2058706us) ts_tablet_manager.cc:1151] Deleting tablet 5930402db43740b4b291c6c238ecf52c
0926 19:58:55.308312 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.308577 (+ 265us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2031845}
I20260926 19:58:55.351730 13403 tablet_replica.cc:333] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:55.358430 13717 raft_consensus.cc:2749] T 46bdb479c5384d0891db66943ef263b2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:55.367967 12641 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
I20260926 19:58:55.369180 13203 ts_tablet_manager.cc:1929] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:55.369547 13203 log.cc:1199] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/d48ee26c0aaa4abc8e08a1a2dadb028d
I20260926 19:58:55.370244 13203 ts_tablet_manager.cc:1950] T d48ee26c0aaa4abc8e08a1a2dadb028d P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:55.370122 13092 raft_consensus.cc:359] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.373383 13092 raft_consensus.cc:385] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.373692 13092 raft_consensus.cc:740] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.373847 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 274) took 1155 ms. Trace:
I20260926 19:58:55.374130 13203 rpcz_store.cc:276] 0926 19:58:54.218868 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.220884 (+ 2016us) service_pool.cc:224] Handling call
0926 19:58:55.253767 (+1032883us) ts_tablet_manager.cc:1151] Deleting tablet d48ee26c0aaa4abc8e08a1a2dadb028d
0926 19:58:55.253807 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.311379 (+ 57572us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.369045 (+ 57666us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.373796 (+ 4751us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1032249}
I20260926 19:58:55.374325 13092 consensus_queue.cc:260] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.375104 13569 ts_tablet_manager.cc:1929] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.375442 13569 log.cc:1199] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/c5b734f0d33342d3be8b1827bcd5dd14
I20260926 19:58:55.375327 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet d48ee26c0aaa4abc8e08a1a2dadb028d (table default.Table_1e071d1293234cecb72d6a118e8d5f10 [id=21d0c2efe78e438e8fad11d811c463ed]) successfully deleted
I20260926 19:58:55.376102 13569 ts_tablet_manager.cc:1950] T c5b734f0d33342d3be8b1827bcd5dd14 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:55.376466 13092 ts_tablet_manager.cc:1434] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260926 19:58:55.377336 13092 tablet_bootstrap.cc:492] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:55.389755 12970 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
I20260926 19:58:55.397534 13092 tablet_bootstrap.cc:654] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.401149 13203 tablet_replica.cc:333] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:55.405403 13403 raft_consensus.cc:2243] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:55.406030 13403 raft_consensus.cc:2272] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:55.406221 13203 raft_consensus.cc:2243] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:55.409039 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet c5b734f0d33342d3be8b1827bcd5dd14 (table default.Table_2d039187041243c8adf2651cf58f0b4a [id=dba288eafec847baaf3732ce6d4292b5]) successfully deleted
I20260926 19:58:55.406672 13203 raft_consensus.cc:2272] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:55.351154 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 5930402db43740b4b291c6c238ecf52c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5930402db43740b4b291c6c238ecf52c already in progress: creating tablet
W20260926 19:58:55.414710 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 986a28107bd54199bf2cda96978150c3 consensus state has no leader
I20260926 19:58:55.416627 13203 ts_tablet_manager.cc:1916] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.423821 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.424458 12564 tablet_service.cc:1558] Processing DeleteTablet for tablet cd86f01d82d041f4852f83f2547e697f with delete_type TABLET_DATA_DELETED (replaced by tablet 3488f0076b0f4ef19f7cf23a93cd20e3) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.425015 12557 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f268b4a1f7423f917731c6b62aab3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.425648 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 with delete_type TABLET_DATA_DELETED (replaced by tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.428007 13092 tablet_bootstrap.cc:492] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:55.428401 13092 ts_tablet_manager.cc:1403] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.000s
W20260926 19:58:55.410082 12382 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 58960us
I20260926 19:58:55.430733 13092 raft_consensus.cc:359] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.431432 13092 raft_consensus.cc:385] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.431689 13092 raft_consensus.cc:740] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.432163 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a1fcffb35ba74139a265e901aba3ae6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:55.432293 13092 consensus_queue.cc:260] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:55.437616 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1fcffb35ba74139a265e901aba3ae6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:55.439654 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d with delete_type TABLET_DATA_DELETED (replaced by tablet 7645307e0999482bbe3329c8d61cc567) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.440066 13092 ts_tablet_manager.cc:1434] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260926 19:58:55.441841 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet 16897e7489e9445ba5389a4f3507a739 with delete_type TABLET_DATA_DELETED (replaced by tablet 986a28107bd54199bf2cda96978150c3) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.443830 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:44 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.445531 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:55.446622 12395 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 57) took 1112 ms (client timeout 1000 ms). Trace:
I20260926 19:58:55.452217 13203 ts_tablet_manager.cc:1929] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.452589 13203 log.cc:1199] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/89c8cbb097a644218e0ca033a6fdb4a2
I20260926 19:58:55.453208 13203 ts_tablet_manager.cc:1950] T 89c8cbb097a644218e0ca033a6fdb4a2 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:55.458700 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_7c45f0a5bb2f469c88cc36ea50f3affe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:55.460258 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c45f0a5bb2f469c88cc36ea50f3affe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:55.459213 13203 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37792 (request call id 277) took 1238 ms. Trace:
I20260926 19:58:55.461687 13203 rpcz_store.cc:276] 0926 19:58:54.221075 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.221625 (+ 550us) service_pool.cc:224] Handling call
0926 19:58:55.400996 (+1179371us) ts_tablet_manager.cc:1151] Deleting tablet 89c8cbb097a644218e0ca033a6fdb4a2
0926 19:58:55.401040 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.428898 (+ 27858us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.452098 (+ 23200us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:55.459154 (+ 7056us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1178869}
I20260926 19:58:55.462865 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 89c8cbb097a644218e0ca033a6fdb4a2 (table default.Table_27371e4eb0cd4fa48aaef20d2db04fcc [id=f5d5844ddbeb4542b5e7b02d22800604]) successfully deleted
W20260926 19:58:55.470654 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 0df471a7b6554d4fb5007adb63babe43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0df471a7b6554d4fb5007adb63babe43 already in progress: creating tablet
W20260926 19:58:55.472182 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet d48ee26c0aaa4abc8e08a1a2dadb028d because the tablet was not found. No further retry: Not found: Tablet not found: d48ee26c0aaa4abc8e08a1a2dadb028d
W20260926 19:58:55.473667 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet cd86f01d82d041f4852f83f2547e697f because the tablet was not found. No further retry: Not found: Tablet not found: cd86f01d82d041f4852f83f2547e697f
I20260926 19:58:55.474471 13203 tablet_replica.cc:333] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:55.475198 13203 raft_consensus.cc:2243] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:55.475639 13203 raft_consensus.cc:2272] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:55.447008 12395 rpcz_store.cc:269] 0926 19:58:54.333953 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.364091 (+ 30138us) service_pool.cc:224] Handling call
0926 19:58:55.446558 (+1082467us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:54.865780 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:54.866157 (+ 377us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:54.866179 (+ 22us) write_op.cc:435] Acquired schema lock
0926 19:58:54.866191 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:54.866452 (+ 261us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:54.866497 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:54.866523 (+ 26us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:54.866677 (+ 154us) tablet.cc:701] Row locks acquired
0926 19:58:54.866695 (+ 18us) write_op.cc:260] PREPARE: finished
0926 19:58:54.866801 (+ 106us) write_op.cc:270] Start()
0926 19:58:54.866874 (+ 73us) write_op.cc:276] Timestamp: P: 1790452734866727 usec, L: 0
0926 19:58:54.866889 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:54.867341 (+ 452us) log.cc:844] Serialized 356 byte log entry
0926 19:58:55.032189 (+164848us) op_driver.cc:464] REPLICATION: finished
0926 19:58:55.034540 (+ 2351us) write_op.cc:301] APPLY: starting
0926 19:58:55.034586 (+ 46us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:55.034680 (+ 94us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:55.034764 (+ 84us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:55.034934 (+ 170us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:55.035048 (+ 114us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:55.035088 (+ 40us) write_op.cc:312] APPLY: finished
0926 19:58:55.035244 (+ 156us) log.cc:844] Serialized 23 byte log entry
0926 19:58:55.035378 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:55.035431 (+ 53us) write_op.cc:454] Released schema lock
0926 19:58:55.035479 (+ 48us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":73441,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":2218,"apply.run_cpu_time_us":1110,"apply.run_wall_time_us":1110,"num_ops":1,"prepare.queue_time_us":6969,"prepare.run_cpu_time_us":1757,"prepare.run_wall_time_us":2751,"replication_time_us":165225,"thread_start_us":85201,"threads_started":1}]]}
I20260926 19:58:55.477339 13203 ts_tablet_manager.cc:1916] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.491195 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet d734112eca1249559d0ff3d123e82042
I20260926 19:58:55.492630 12396 catalog_manager.cc:6934] ReplaceTablet: tablet d734112eca1249559d0ff3d123e82042 deleted and replaced by tablet 9ed22219df2b42ad86a6d7917ee0d990
I20260926 19:58:55.495273 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet d734112eca1249559d0ff3d123e82042 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ed22219df2b42ad86a6d7917ee0d990) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:55.498291 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet d734112eca1249559d0ff3d123e82042 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d734112eca1249559d0ff3d123e82042 already in progress: creating tablet
I20260926 19:58:55.502655 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet d734112eca1249559d0ff3d123e82042 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ed22219df2b42ad86a6d7917ee0d990) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:55.504431 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet d734112eca1249559d0ff3d123e82042 with delete_type TABLET_DATA_DELETED (replaced by tablet 9ed22219df2b42ad86a6d7917ee0d990) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.510984 13203 ts_tablet_manager.cc:1929] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.511354 13203 log.cc:1199] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/22f268b4a1f7423f917731c6b62aab3e
I20260926 19:58:55.511996 13203 ts_tablet_manager.cc:1950] T 22f268b4a1f7423f917731c6b62aab3e P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:55.517416 13013 tablet_bootstrap.cc:492] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:55.518428 13013 ts_tablet_manager.cc:1403] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.503s user 0.010s sys 0.000s
I20260926 19:58:55.525862 13203 tablet_replica.cc:333] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:55.526613 13203 raft_consensus.cc:2243] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:55.527021 13203 raft_consensus.cc:2272] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:55.530252 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 22f268b4a1f7423f917731c6b62aab3e (table default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]) successfully deleted
I20260926 19:58:55.531766 13203 ts_tablet_manager.cc:1916] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.541785 13403 ts_tablet_manager.cc:1916] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.545522 13012 tablet_bootstrap.cc:492] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:55.546041 13012 ts_tablet_manager.cc:1403] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.413s user 0.008s sys 0.003s
I20260926 19:58:55.549964 13012 raft_consensus.cc:359] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.550951 13012 raft_consensus.cc:385] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.551273 13012 raft_consensus.cc:740] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.552276 13012 consensus_queue.cc:260] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.553828 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_6787fdb6bfad44bc8eb2f6c7c0848f48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:55.555513 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6787fdb6bfad44bc8eb2f6c7c0848f48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:55.560014 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_d20824670f7d4516b7387bc0d28da6dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:55.561712 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d20824670f7d4516b7387bc0d28da6dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:55.577973 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet bfdfc9dd1b6046d18459a1255be03c4c
I20260926 19:58:55.582685 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet bfdfc9dd1b6046d18459a1255be03c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.584374 13203 ts_tablet_manager.cc:1929] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.584721 13203 log.cc:1199] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/b32f8b0d480547a58ba5c6fed6483a74
I20260926 19:58:55.585297 13203 ts_tablet_manager.cc:1950] T b32f8b0d480547a58ba5c6fed6483a74 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:55.595014 12393 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_15cc720c12e6419d8ca44d8485d7c80b" } new_table_name: "default.Table_07843c7a37754743a74a6b3b822060a6" modify_external_catalogs: true
I20260926 19:58:55.597892 13203 tablet_replica.cc:333] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:55.597806 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet b32f8b0d480547a58ba5c6fed6483a74 (table default.Table_1cd57fde79174a2e8092337be2e108c1 [id=f15adbe1da374a8b9b507ff371c1286c]) successfully deleted
I20260926 19:58:55.598582 13203 raft_consensus.cc:2243] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:55.598994 13203 raft_consensus.cc:2272] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:55.600518 13203 ts_tablet_manager.cc:1916] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:55.615823 13203 ts_tablet_manager.cc:1929] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:55.616163 13203 log.cc:1199] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/acdb4bd8cd58444cb1d6d4b5610de20d
I20260926 19:58:55.616780 13203 ts_tablet_manager.cc:1950] T acdb4bd8cd58444cb1d6d4b5610de20d P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:55.618368 12391 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet deb7ea3bde5e4464a48b6e2843599f7d
I20260926 19:58:55.619031 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet acdb4bd8cd58444cb1d6d4b5610de20d (table default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]) successfully deleted
W20260926 19:58:55.620469 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 16897e7489e9445ba5389a4f3507a739 because the tablet was not found. No further retry: Not found: Tablet not found: 16897e7489e9445ba5389a4f3507a739
W20260926 19:58:55.621812 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 06c5ef9e1bca427fb889cbb17f77f999 because the tablet was not found. No further retry: Not found: Tablet not found: 06c5ef9e1bca427fb889cbb17f77f999
I20260926 19:58:55.623504 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet deb7ea3bde5e4464a48b6e2843599f7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:55.624933 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet c5b734f0d33342d3be8b1827bcd5dd14 because the tablet was not found. No further retry: Not found: Tablet not found: c5b734f0d33342d3be8b1827bcd5dd14
W20260926 19:58:55.626375 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet d734112eca1249559d0ff3d123e82042 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d734112eca1249559d0ff3d123e82042 already in progress: creating tablet
W20260926 19:58:55.627882 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet bfdfc9dd1b6046d18459a1255be03c4c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfdfc9dd1b6046d18459a1255be03c4c already in progress: creating tablet
W20260926 19:58:55.628890 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet deb7ea3bde5e4464a48b6e2843599f7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet deb7ea3bde5e4464a48b6e2843599f7d already in progress: creating tablet
I20260926 19:58:55.543474 13092 tablet_bootstrap.cc:492] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:55.581799 13717 raft_consensus.cc:493] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.584162 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet bfdfc9dd1b6046d18459a1255be03c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:55.620103 12893 tablet_service.cc:1558] Processing DeleteTablet for tablet deb7ea3bde5e4464a48b6e2843599f7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:55.633742 13717 raft_consensus.cc:515] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.635692 13717 leader_election.cc:290] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:55.639115 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0583424ef64d9ea3beeb1c01ceafba" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:55.639703 12594 raft_consensus.cc:2468] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:55.641224 12858 leader_election.cc:304] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:55.642059 13717 raft_consensus.cc:2804] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:55.642385 13717 raft_consensus.cc:493] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:55.642669 13717 raft_consensus.cc:3060] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:55.648293 13092 tablet_bootstrap.cc:654] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.654706 13717 raft_consensus.cc:515] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.659788 13724 raft_consensus.cc:493] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.660291 13724 raft_consensus.cc:515] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.662256 13724 leader_election.cc:290] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:55.665807 12589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0583424ef64d9ea3beeb1c01ceafba" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:55.666096 13717 leader_election.cc:290] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:55.666378 12589 raft_consensus.cc:3060] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:55.669710 13830 log.cc:927] Time spent T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Append to log took a long time: real 0.131s user 0.002s sys 0.000s
I20260926 19:58:55.670514 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5930402db43740b4b291c6c238ecf52c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:55.671166 13092 tablet_bootstrap.cc:492] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:55.671579 13092 ts_tablet_manager.cc:1403] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.128s user 0.004s sys 0.008s
W20260926 19:58:55.672564 12521 leader_election.cc:343] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.674014 13092 raft_consensus.cc:359] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.674778 13092 raft_consensus.cc:385] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.675024 13092 raft_consensus.cc:740] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.520954 13013 raft_consensus.cc:359] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.675642 13092 consensus_queue.cc:260] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.676229 13013 raft_consensus.cc:385] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.676522 13013 raft_consensus.cc:740] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.677170 13013 consensus_queue.cc:260] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.679579 12589 raft_consensus.cc:2468] T 1d0583424ef64d9ea3beeb1c01ceafba P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:55.680761 12858 leader_election.cc:304] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:55.681523 13803 raft_consensus.cc:2804] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:55.681927 13803 raft_consensus.cc:697] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:55.682907 13092 ts_tablet_manager.cc:1434] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260926 19:58:55.682683 13803 consensus_queue.cc:237] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.683756 13092 tablet_bootstrap.cc:492] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:55.684041 13013 ts_tablet_manager.cc:1434] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.165s user 0.005s sys 0.000s
I20260926 19:58:55.684863 13013 tablet_bootstrap.cc:492] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:55.687052 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_17815f4814b9429da94beee1f7a60322" } new_table_name: "default.Table_2d59ce3a076646599fc1d012d3495115" modify_external_catalogs: true
I20260926 19:58:55.688971 13092 tablet_bootstrap.cc:654] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.581014 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet bfdfc9dd1b6046d18459a1255be03c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:55.594106 13840 raft_consensus.cc:493] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.621865 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet deb7ea3bde5e4464a48b6e2843599f7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:55.704741 13840 raft_consensus.cc:515] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.637257 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0583424ef64d9ea3beeb1c01ceafba" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:55.706702 13012 ts_tablet_manager.cc:1434] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.160s user 0.000s sys 0.005s
I20260926 19:58:55.663582 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5930402db43740b4b291c6c238ecf52c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:55.672305 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0583424ef64d9ea3beeb1c01ceafba" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
W20260926 19:58:55.707854 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet bfdfc9dd1b6046d18459a1255be03c4c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfdfc9dd1b6046d18459a1255be03c4c already in progress: creating tablet
I20260926 19:58:55.708124 13012 tablet_bootstrap.cc:492] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:55.708518 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet deb7ea3bde5e4464a48b6e2843599f7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet deb7ea3bde5e4464a48b6e2843599f7d already in progress: creating tablet
W20260926 19:58:55.708640 12857 leader_election.cc:343] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.709583 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acdb4bd8cd58444cb1d6d4b5610de20d" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:55.710464 12519 leader_election.cc:343] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.710843 12519 leader_election.cc:304] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
W20260926 19:58:55.711269 12696 leader_election.cc:343] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.711555 13724 raft_consensus.cc:2749] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:55.711951 12857 leader_election.cc:343] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.706712 13840 leader_election.cc:290] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:55.713013 12807 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
W20260926 19:58:55.713922 12695 leader_election.cc:343] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: acdb4bd8cd58444cb1d6d4b5610de20d
I20260926 19:58:55.714291 12695 leader_election.cc:304] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:55.715153 13840 raft_consensus.cc:2749] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:55.729590 12641 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
I20260926 19:58:55.736069 13724 raft_consensus.cc:493] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.736549 13724 raft_consensus.cc:515] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.738437 13724 leader_election.cc:290] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:55.739982 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:55.740072 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:55.741443 12519 leader_election.cc:343] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:55.741967 12521 leader_election.cc:343] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:55.742324 12521 leader_election.cc:304] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:55.743227 13724 raft_consensus.cc:2749] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:55.746935 13092 tablet_bootstrap.cc:492] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:55.747339 13092 ts_tablet_manager.cc:1403] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.064s user 0.011s sys 0.009s
I20260926 19:58:55.749601 13092 raft_consensus.cc:359] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.750205 13092 raft_consensus.cc:385] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.750456 13092 raft_consensus.cc:740] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.751035 13092 consensus_queue.cc:260] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.753167 13092 ts_tablet_manager.cc:1434] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260926 19:58:55.754024 13092 tablet_bootstrap.cc:492] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:55.759359 13092 tablet_bootstrap.cc:654] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:55.781646 13840 raft_consensus.cc:493] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.782119 13840 raft_consensus.cc:515] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.784114 13840 leader_election.cc:290] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:55.784766 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet de85e3e17bf340f8895aff3f5421c85c
I20260926 19:58:55.786131 12387 catalog_manager.cc:6934] ReplaceTablet: tablet de85e3e17bf340f8895aff3f5421c85c deleted and replaced by tablet 8fb9c1e977f942d9b6e3a00c3f4bc694
W20260926 19:58:55.786381 12695 leader_election.cc:343] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: cd86f01d82d041f4852f83f2547e697f
I20260926 19:58:55.788002 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd86f01d82d041f4852f83f2547e697f" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:55.788600 12918 raft_consensus.cc:2468] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:55.789857 12696 leader_election.cc:304] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:55.790721 12893 tablet_service.cc:1558] Processing DeleteTablet for tablet de85e3e17bf340f8895aff3f5421c85c with delete_type TABLET_DATA_DELETED (replaced by tablet 8fb9c1e977f942d9b6e3a00c3f4bc694) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:55.790938 13840 raft_consensus.cc:2804] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:55.791239 13840 raft_consensus.cc:493] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:55.791370 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet de85e3e17bf340f8895aff3f5421c85c with delete_type TABLET_DATA_DELETED (replaced by tablet 8fb9c1e977f942d9b6e3a00c3f4bc694) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:55.791544 13840 raft_consensus.cc:3060] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:55.792913 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet de85e3e17bf340f8895aff3f5421c85c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de85e3e17bf340f8895aff3f5421c85c already in progress: creating tablet
I20260926 19:58:55.800381 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet de85e3e17bf340f8895aff3f5421c85c with delete_type TABLET_DATA_DELETED (replaced by tablet 8fb9c1e977f942d9b6e3a00c3f4bc694) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:55.803043 13833 consensus_queue.cc:1048] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:55.811753 13840 raft_consensus.cc:515] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.815025 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd86f01d82d041f4852f83f2547e697f" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:55.815569 12918 raft_consensus.cc:3060] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:55.823979 12918 raft_consensus.cc:2468] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:55.825594 12696 leader_election.cc:304] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:55.826931 13833 raft_consensus.cc:2804] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:55.827564 13833 raft_consensus.cc:697] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:55.828446 13833 consensus_queue.cc:237] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.833092 13840 leader_election.cc:290] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:55.834419 13092 tablet_bootstrap.cc:492] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:55.834817 13092 ts_tablet_manager.cc:1403] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.081s user 0.014s sys 0.008s
W20260926 19:58:55.843650 12695 leader_election.cc:343] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: cd86f01d82d041f4852f83f2547e697f
I20260926 19:58:55.848109 13092 raft_consensus.cc:359] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:55.855654 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet de85e3e17bf340f8895aff3f5421c85c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de85e3e17bf340f8895aff3f5421c85c already in progress: creating tablet
I20260926 19:58:55.854372 13092 raft_consensus.cc:385] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:55.858767 13092 raft_consensus.cc:740] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:55.860606 13092 consensus_queue.cc:260] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.866925 13092 ts_tablet_manager.cc:1434] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.025s user 0.005s sys 0.001s
I20260926 19:58:55.884581 13840 raft_consensus.cc:493] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:55.885622 13853 consensus_queue.cc:1048] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:58:55.885251 13840 raft_consensus.cc:515] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:55.920401 12387 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9277497973f14dc6a9ebfdb45308c646" } modify_external_catalogs: true
I20260926 19:58:55.921123 12387 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_9277497973f14dc6a9ebfdb45308c646" } modify_external_catalogs: true
W20260926 19:58:56.009495 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 3df392d277a447aabf4320a7d9a43880 (table default.Table_82df428358de4fc18b05ae70c217854b [id=f4a644d228fe4e47985e6142a522275d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3df392d277a447aabf4320a7d9a43880 consensus state has no leader
I20260926 19:58:56.023535 13092 tablet_bootstrap.cc:492] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:56.030272 13092 tablet_bootstrap.cc:654] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.043876 13012 tablet_bootstrap.cc:700] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent opening tablet: real 0.208s user 0.000s sys 0.000s
I20260926 19:58:56.048434 13092 tablet_bootstrap.cc:492] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:56.048839 13092 ts_tablet_manager.cc:1403] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.007s
I20260926 19:58:56.055961 13830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260926 19:58:56.057053 13830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Bootstrap complete.
I20260926 19:58:56.067227 12389 master_service.cc:960] ReplaceTablet: received request to replace tablet 90b7a9622b0f412b9a88aa84419ad0d7 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:56.077054 13013 tablet_bootstrap.cc:654] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.076925 13092 raft_consensus.cc:359] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.077816 13092 raft_consensus.cc:385] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:56.078145 13092 raft_consensus.cc:740] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:56.045674 13840 leader_election.cc:290] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:58:56.049934 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 206334us
I20260926 19:58:56.090372 13012 tablet_bootstrap.cc:654] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.094220 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "791b4ef9be004d7ead26591c50c6f405" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:56.094877 12593 raft_consensus.cc:2468] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:56.098961 13840 raft_consensus.cc:493] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:56.099418 13840 raft_consensus.cc:515] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.105712 13403 ts_tablet_manager.cc:1929] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:56.105865 13840 leader_election.cc:290] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:56.106113 13403 log.cc:1199] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/fb593ce2cd4147febebfe442de9fdd28
I20260926 19:58:56.107165 13803 raft_consensus.cc:493] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:56.107599 13803 raft_consensus.cc:515] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.109840 12695 leader_election.cc:304] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
W20260926 19:58:56.111711 12858 leader_election.cc:343] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 89c8cbb097a644218e0ca033a6fdb4a2
I20260926 19:58:56.111821 13861 raft_consensus.cc:2804] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:56.112972 13861 raft_consensus.cc:493] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:56.113303 13803 leader_election.cc:290] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:56.113361 13861 raft_consensus.cc:3060] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:56.118317 13403 ts_tablet_manager.cc:1950] T fb593ce2cd4147febebfe442de9fdd28 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:56.118711 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "791b4ef9be004d7ead26591c50c6f405" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:56.119407 12918 raft_consensus.cc:2468] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
W20260926 19:58:56.119640 12695 leader_election.cc:343] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 06c5ef9e1bca427fb889cbb17f77f999
I20260926 19:58:56.120350 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 243) took 2435 ms. Trace:
I20260926 19:58:56.120654 13403 rpcz_store.cc:276] 0926 19:58:53.685267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.685406 (+ 139us) service_pool.cc:224] Handling call
0926 19:58:55.351586 (+1666180us) ts_tablet_manager.cc:1151] Deleting tablet fb593ce2cd4147febebfe442de9fdd28
0926 19:58:55.351629 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:55.725651 (+374022us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:56.084959 (+359308us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:56.120295 (+ 35336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52785,"tablet-delete.queue_time_us":1620837}
I20260926 19:58:56.129199 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_724fda2a18204a24979779d6e0aafb81" } modify_external_catalogs: true
I20260926 19:58:56.129761 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_724fda2a18204a24979779d6e0aafb81" } modify_external_catalogs: true
I20260926 19:58:56.134923 13013 tablet_bootstrap.cc:492] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:56.135398 13013 ts_tablet_manager.cc:1403] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.451s user 0.014s sys 0.000s
I20260926 19:58:56.137768 13013 raft_consensus.cc:359] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:56.150895 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89c8cbb097a644218e0ca033a6fdb4a2" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:56.163187 12396 master_service.cc:960] ReplaceTablet: received request to replace tablet 4d153e60ae5d486cb91c2cc623b99985 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:56.171377 12353 master_runner.cc:187] Error getting master registration for 127.11.233.253:37913: OK, instance_id { permanent_uuid: "db2490adb6e84090b780f52470e580ab" instance_seqno: 1790452734722749 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:58:56.172438 12353 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 19:58:56.187171 12393 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb5e54722cfe43e5ada9feafaaa23964 consensus state has no leader
W20260926 19:58:56.189476 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d4434cb08d0f41fd96283de49c53b3e8 consensus state has no leader
I20260926 19:58:56.144481 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c5ef9e1bca427fb889cbb17f77f999" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:56.161805 12857 leader_election.cc:343] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:56.206550 12920 raft_consensus.cc:2468] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:56.206959 12857 leader_election.cc:304] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:56.207872 13403 tablet_replica.cc:333] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
W20260926 19:58:56.208052 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 46293.3us
I20260926 19:58:56.208613 13403 raft_consensus.cc:2243] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:56.209071 13403 raft_consensus.cc:2272] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:56.210493 13803 raft_consensus.cc:2749] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:56.210991 13403 ts_tablet_manager.cc:1916] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:56.217873 13013 raft_consensus.cc:385] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:56.218271 13013 raft_consensus.cc:740] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:56.219059 13013 consensus_queue.cc:260] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:56.224565 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet fb593ce2cd4147febebfe442de9fdd28 (table default.Table_693c5b4b4dc446378398965b02ddeea5 [id=d088bdbe9cf34c41adfc15548ca5521a]) successfully deleted
I20260926 19:58:56.231911 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet caf4547f148f49b8943f4e1c7f2132a7
I20260926 19:58:56.234557 12558 tablet_service.cc:1558] Processing DeleteTablet for tablet caf4547f148f49b8943f4e1c7f2132a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:56 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:56.235671 13203 tablet_replica.cc:333] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:56.235580 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet caf4547f148f49b8943f4e1c7f2132a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:56 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.236336 13203 raft_consensus.cc:2243] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:56.235805 12893 tablet_service.cc:1558] Processing DeleteTablet for tablet caf4547f148f49b8943f4e1c7f2132a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:56 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:56.236779 13203 raft_consensus.cc:2272] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:56.237737 13569 tablet_replica.cc:333] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:56.238431 13569 raft_consensus.cc:2243] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:56.238809 13203 ts_tablet_manager.cc:1916] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:56.244374 13013 ts_tablet_manager.cc:1434] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.109s user 0.005s sys 0.000s
I20260926 19:58:56.245199 13013 tablet_bootstrap.cc:492] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
W20260926 19:58:56.249732 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 43881.2us
I20260926 19:58:56.255825 13569 raft_consensus.cc:2272] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:56.257699 13569 ts_tablet_manager.cc:1916] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:56.261168 13856 raft_consensus.cc:493] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:56.261610 13856 raft_consensus.cc:515] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:56.263414 13856 leader_election.cc:290] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:56.265695 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7dc92330d400f93f427f3f79ad826" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:56.173808 13092 consensus_queue.cc:260] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.269419 13092 ts_tablet_manager.cc:1434] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.220s user 0.007s sys 0.000s
W20260926 19:58:56.269552 12521 leader_election.cc:343] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:56.273123 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_4af5dbf295ac46afa888532ae4a8777f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:56.274595 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4af5dbf295ac46afa888532ae4a8777f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:56.278082 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_2079ba9cd2c7475a8046395706e15d9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:56.279435 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2079ba9cd2c7475a8046395706e15d9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:56.253399 12696 leader_election.cc:304] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:56.283288 13092 tablet_bootstrap.cc:492] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:56.288398 13092 tablet_bootstrap.cc:654] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.297089 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7dc92330d400f93f427f3f79ad826" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:56.295964 13840 raft_consensus.cc:2804] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:56.298813 13840 raft_consensus.cc:493] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:56.299103 13840 raft_consensus.cc:3060] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:56.299059 12519 leader_election.cc:343] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:56.299423 12519 leader_election.cc:304] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:56.300491 13724 raft_consensus.cc:2749] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:56.301597 12696 consensus_peers.cc:597] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:56.308570 13569 ts_tablet_manager.cc:1929] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:56.308913 13569 log.cc:1199] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/caf4547f148f49b8943f4e1c7f2132a7
I20260926 19:58:56.309613 13569 ts_tablet_manager.cc:1950] T caf4547f148f49b8943f4e1c7f2132a7 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:56.310926 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet caf4547f148f49b8943f4e1c7f2132a7 (table default.Table_724fda2a18204a24979779d6e0aafb81 [id=b89cc0604f6f4019802a9ce497b4b954]) successfully deleted
I20260926 19:58:56.314706 13203 ts_tablet_manager.cc:1929] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:56.315064 13203 log.cc:1199] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/caf4547f148f49b8943f4e1c7f2132a7
I20260926 19:58:56.322781 13861 raft_consensus.cc:515] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.325774 12593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "791b4ef9be004d7ead26591c50c6f405" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:56.326519 12593 raft_consensus.cc:3060] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:56.328547 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "791b4ef9be004d7ead26591c50c6f405" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:56.329124 12918 raft_consensus.cc:3060] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:56.336540 12918 raft_consensus.cc:2468] T 791b4ef9be004d7ead26591c50c6f405 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:56.342996 12392 catalog_manager.cc:5838] T 46bdb479c5384d0891db66943ef263b2 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:56.344012 13012 tablet_bootstrap.cc:492] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:56.344425 13012 ts_tablet_manager.cc:1403] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.637s user 0.014s sys 0.006s
I20260926 19:58:56.344935 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:56.346741 12696 leader_election.cc:304] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:56.315722 13203 ts_tablet_manager.cc:1950] T caf4547f148f49b8943f4e1c7f2132a7 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:56.349036 12893 tablet_service.cc:1511] Processing CreateTablet for tablet 53af06be466d47569d5aa15727dadd0a (DEFAULT_TABLE table=default.Table_9277497973f14dc6a9ebfdb45308c646 [id=bbcd8fe7d4ba4a4cb3fec461cd9c5f31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.350514 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet caf4547f148f49b8943f4e1c7f2132a7 (table default.Table_724fda2a18204a24979779d6e0aafb81 [id=b89cc0604f6f4019802a9ce497b4b954]) successfully deleted
I20260926 19:58:56.351158 12893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53af06be466d47569d5aa15727dadd0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.355646 13871 raft_consensus.cc:493] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:56.356086 13871 raft_consensus.cc:515] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.345449 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:58:56.358567 13871 leader_election.cc:290] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:58:56.359731 12858 leader_election.cc:343] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: acdb4bd8cd58444cb1d6d4b5610de20d
I20260926 19:58:56.360502 12593 raft_consensus.cc:2468] T 791b4ef9be004d7ead26591c50c6f405 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:56.364669 13840 raft_consensus.cc:515] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.365567 13012 raft_consensus.cc:359] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.366187 13012 raft_consensus.cc:385] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:56.366439 13012 raft_consensus.cc:740] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:56.367496 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c5ef9e1bca427fb889cbb17f77f999" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:56.368062 12918 raft_consensus.cc:3060] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:56.369153 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_564a067586d540a6919f287941cf0392"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:56.370615 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_564a067586d540a6919f287941cf0392 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:56.372994 13860 raft_consensus.cc:2804] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:56.373471 13860 raft_consensus.cc:697] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:56.374270 13860 consensus_queue.cc:237] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.374605 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_94fb4afa42c74fa1b7ec938574559b46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:56.376032 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94fb4afa42c74fa1b7ec938574559b46 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:56.378803 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 76449.9us
I20260926 19:58:56.387281 12918 raft_consensus.cc:2468] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:56.390702 13012 consensus_queue.cc:260] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.393287 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acdb4bd8cd58444cb1d6d4b5610de20d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:56.393864 12759 raft_consensus.cc:2468] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:56.394305 12696 leader_election.cc:304] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:56.396457 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 53af06be466d47569d5aa15727dadd0a (DEFAULT_TABLE table=default.Table_9277497973f14dc6a9ebfdb45308c646 [id=bbcd8fe7d4ba4a4cb3fec461cd9c5f31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.397466 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53af06be466d47569d5aa15727dadd0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.398522 12558 tablet_service.cc:1511] Processing CreateTablet for tablet 53af06be466d47569d5aa15727dadd0a (DEFAULT_TABLE table=default.Table_9277497973f14dc6a9ebfdb45308c646 [id=bbcd8fe7d4ba4a4cb3fec461cd9c5f31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.398986 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 4d153e60ae5d486cb91c2cc623b99985
I20260926 19:58:56.346261 13861 leader_election.cc:290] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:56.400290 12396 catalog_manager.cc:6934] ReplaceTablet: tablet 4d153e60ae5d486cb91c2cc623b99985 deleted and replaced by tablet a1d5deb0364542a7945e5675ca31557c
I20260926 19:58:56.400974 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_82df428358de4fc18b05ae70c217854b" } new_table_name: "default.Table_71b660a4d8c04aae800e0747908f556e" modify_external_catalogs: true
I20260926 19:58:56.402542 12731 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9e9e5038774f6cb35de166f1fd78d4 (DEFAULT_TABLE table=default.Table_785a1ec64e194c679e03b352eb694de4 [id=5175876a958c45eaa5d0a03986c9f23c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.403579 12731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9e9e5038774f6cb35de166f1fd78d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.405280 12896 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9e9e5038774f6cb35de166f1fd78d4 (DEFAULT_TABLE table=default.Table_785a1ec64e194c679e03b352eb694de4 [id=5175876a958c45eaa5d0a03986c9f23c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.406315 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9e9e5038774f6cb35de166f1fd78d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.408317 12890 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d153e60ae5d486cb91c2cc623b99985 with delete_type TABLET_DATA_DELETED (replaced by tablet a1d5deb0364542a7945e5675ca31557c) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:56.411741 13013 tablet_bootstrap.cc:654] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.418208 12557 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d153e60ae5d486cb91c2cc623b99985 with delete_type TABLET_DATA_DELETED (replaced by tablet a1d5deb0364542a7945e5675ca31557c) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:56.419250 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 5d9e9e5038774f6cb35de166f1fd78d4 (DEFAULT_TABLE table=default.Table_785a1ec64e194c679e03b352eb694de4 [id=5175876a958c45eaa5d0a03986c9f23c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.428309 12890 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea31ef81ae44318b06d8af0c015cd86 (DEFAULT_TABLE table=default.Table_b359ca01127641a4b703f455ec9bee7b [id=9bb5faedd21948d6933db3f22744d0de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.429368 12890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea31ef81ae44318b06d8af0c015cd86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.430312 13853 raft_consensus.cc:2804] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:56.431210 13853 raft_consensus.cc:697] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:56.431347 13012 ts_tablet_manager.cc:1434] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.087s user 0.006s sys 0.001s
I20260926 19:58:56.432185 13012 tablet_bootstrap.cc:492] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:56.431954 13853 consensus_queue.cc:237] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.434710 12729 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d153e60ae5d486cb91c2cc623b99985 with delete_type TABLET_DATA_DELETED (replaced by tablet a1d5deb0364542a7945e5675ca31557c) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.435827 12730 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea31ef81ae44318b06d8af0c015cd86 (DEFAULT_TABLE table=default.Table_b359ca01127641a4b703f455ec9bee7b [id=9bb5faedd21948d6933db3f22744d0de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.435978 13859 consensus_queue.cc:1048] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:56.436833 12730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea31ef81ae44318b06d8af0c015cd86. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:56.445055 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 4d153e60ae5d486cb91c2cc623b99985 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d153e60ae5d486cb91c2cc623b99985 already in progress: creating tablet
I20260926 19:58:56.449041 12893 tablet_service.cc:1511] Processing CreateTablet for tablet f28394cd29c44af6b8c662f6c9e1165d (DEFAULT_TABLE table=default.Table_beef77b0962d44d8948a5f4984c0956f [id=35a95e8c58c64b73b6a80d93a709d475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.449175 12557 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea31ef81ae44318b06d8af0c015cd86 (DEFAULT_TABLE table=default.Table_b359ca01127641a4b703f455ec9bee7b [id=9bb5faedd21948d6933db3f22744d0de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.450177 12893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28394cd29c44af6b8c662f6c9e1165d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.453086 13724 raft_consensus.cc:493] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260926 19:58:56.453761 12695 leader_election.cc:343] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 06c5ef9e1bca427fb889cbb17f77f999
I20260926 19:58:56.453542 13724 raft_consensus.cc:515] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.455348 13724 leader_election.cc:290] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:56.455361 13840 leader_election.cc:290] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:56.456087 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27c5b06d33c43ab9a483e1fc9b18234" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:56.457489 12521 leader_election.cc:343] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:56.457567 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27c5b06d33c43ab9a483e1fc9b18234" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:56.458819 12519 leader_election.cc:343] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:56.459208 12519 leader_election.cc:304] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:56.460963 13724 raft_consensus.cc:2749] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:56.465049 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53af06be466d47569d5aa15727dadd0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.466148 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d9e9e5038774f6cb35de166f1fd78d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:56.468303 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 74070.7us
I20260926 19:58:56.469168 12857 leader_election.cc:304] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:56.469487 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 53af06be466d47569d5aa15727dadd0a
I20260926 19:58:56.470103 12557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea31ef81ae44318b06d8af0c015cd86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.470172 13803 raft_consensus.cc:2804] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:56.470475 13803 raft_consensus.cc:493] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:56.470783 13803 raft_consensus.cc:3060] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:56.472040 12729 tablet_service.cc:1558] Processing DeleteTablet for tablet 53af06be466d47569d5aa15727dadd0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:56.473866 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 53af06be466d47569d5aa15727dadd0a because the tablet was not found. No further retry: Not found: Tablet not found: 53af06be466d47569d5aa15727dadd0a
I20260926 19:58:56.474802 12895 tablet_service.cc:1558] Processing DeleteTablet for tablet 53af06be466d47569d5aa15727dadd0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:56.476130 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 53af06be466d47569d5aa15727dadd0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:55 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:56.477833 13803 raft_consensus.cc:515] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.480695 12729 tablet_service.cc:1511] Processing CreateTablet for tablet f28394cd29c44af6b8c662f6c9e1165d (DEFAULT_TABLE table=default.Table_beef77b0962d44d8948a5f4984c0956f [id=35a95e8c58c64b73b6a80d93a709d475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.481830 12729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28394cd29c44af6b8c662f6c9e1165d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:56.489827 12858 leader_election.cc:343] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: acdb4bd8cd58444cb1d6d4b5610de20d
I20260926 19:58:56.517242 13013 tablet_bootstrap.cc:492] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:56.517777 13013 ts_tablet_manager.cc:1403] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.273s user 0.005s sys 0.005s
I20260926 19:58:56.523278 12389 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 90b7a9622b0f412b9a88aa84419ad0d7
I20260926 19:58:56.524674 12389 catalog_manager.cc:6934] ReplaceTablet: tablet 90b7a9622b0f412b9a88aa84419ad0d7 deleted and replaced by tablet a0c465b503b54e569fbf529da3ac316c
I20260926 19:58:56.528306 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 90b7a9622b0f412b9a88aa84419ad0d7 with delete_type TABLET_DATA_DELETED (replaced by tablet a0c465b503b54e569fbf529da3ac316c) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:56.534761 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:56.535550 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:56.537156 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
W20260926 19:58:56.537199 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 4d153e60ae5d486cb91c2cc623b99985 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d153e60ae5d486cb91c2cc623b99985 already in progress: creating tablet
W20260926 19:58:56.537912 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 53af06be466d47569d5aa15727dadd0a because the tablet was not found. No further retry: Not found: Tablet not found: 53af06be466d47569d5aa15727dadd0a
W20260926 19:58:56.538528 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 90b7a9622b0f412b9a88aa84419ad0d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90b7a9622b0f412b9a88aa84419ad0d7 already in progress: creating tablet
I20260926 19:58:56.540963 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_5f94ac931883406abfbdd3da405dc80a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:56.542475 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f94ac931883406abfbdd3da405dc80a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:56.546397 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_861452498c1d4bcab2471cd4e97d830a" } modify_external_catalogs: true
I20260926 19:58:56.546892 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_861452498c1d4bcab2471cd4e97d830a" } modify_external_catalogs: true
I20260926 19:58:56.548708 12562 tablet_service.cc:1511] Processing CreateTablet for tablet f28394cd29c44af6b8c662f6c9e1165d (DEFAULT_TABLE table=default.Table_beef77b0962d44d8948a5f4984c0956f [id=35a95e8c58c64b73b6a80d93a709d475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.549708 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f28394cd29c44af6b8c662f6c9e1165d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.554852 12735 tablet_service.cc:1511] Processing CreateTablet for tablet de71cbac8eef423c897c8c142fd29b61 (DEFAULT_TABLE table=default.Table_652bcc921b9c4e77943bc28b1c6e2166 [id=a30267dc46484a46be8980dcebeb4b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.555097 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet 90b7a9622b0f412b9a88aa84419ad0d7 with delete_type TABLET_DATA_DELETED (replaced by tablet a0c465b503b54e569fbf529da3ac316c) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.556088 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de71cbac8eef423c897c8c142fd29b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.558835 13403 ts_tablet_manager.cc:1929] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260926 19:58:56.559015 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3df392d277a447aabf4320a7d9a43880 consensus state has no leader
I20260926 19:58:56.559209 13403 log.cc:1199] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/d48ee26c0aaa4abc8e08a1a2dadb028d
I20260926 19:58:56.559825 13403 ts_tablet_manager.cc:1950] T d48ee26c0aaa4abc8e08a1a2dadb028d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:56.560509 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 246) took 2628 ms. Trace:
I20260926 19:58:56.560806 13403 rpcz_store.cc:276] 0926 19:58:53.931753 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:53.948107 (+ 16354us) service_pool.cc:224] Handling call
0926 19:58:56.207713 (+2259606us) ts_tablet_manager.cc:1151] Deleting tablet d48ee26c0aaa4abc8e08a1a2dadb028d
0926 19:58:56.207758 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:56.403327 (+195569us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:56.558723 (+155396us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:56.560463 (+ 1740us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2259028}
W20260926 19:58:56.562799 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 90b7a9622b0f412b9a88aa84419ad0d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90b7a9622b0f412b9a88aa84419ad0d7 already in progress: creating tablet
I20260926 19:58:56.565928 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49]). Expected version 1 got 0
I20260926 19:58:56.569414 13013 raft_consensus.cc:359] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:56.570182 13013 raft_consensus.cc:385] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:56.570462 13013 raft_consensus.cc:740] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:56.571108 13013 consensus_queue.cc:260] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:56.573670 13013 ts_tablet_manager.cc:1434] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.056s user 0.006s sys 0.000s
I20260926 19:58:56.573237 12391 catalog_manager.cc:5838] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195). New cstate: current_term: 1 leader_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:56.584010 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_81cb6f916c5d49a18d46e44cfe747ee0" } new_table_name: "default.Table_8d453d8c87c949c6b28b98fdf69184d3" modify_external_catalogs: true
I20260926 19:58:56.584787 13012 tablet_bootstrap.cc:654] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.589237 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_da82d46a08d745ddaebd2a7c79b1d4be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:56.590765 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da82d46a08d745ddaebd2a7c79b1d4be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:56.599514 13013 tablet_bootstrap.cc:492] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:56.612759 12561 tablet_service.cc:1511] Processing CreateTablet for tablet de71cbac8eef423c897c8c142fd29b61 (DEFAULT_TABLE table=default.Table_652bcc921b9c4e77943bc28b1c6e2166 [id=a30267dc46484a46be8980dcebeb4b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.614164 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de71cbac8eef423c897c8c142fd29b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.614910 13013 tablet_bootstrap.cc:654] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.620404 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 92a59e9527744e8e99f0b0b20b98e729 (DEFAULT_TABLE table=default.Table_86e17c4523fa4812a508156cff380a35 [id=3bef47b1fe5f4403b2b5e615f7baeaf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.620107 12730 tablet_service.cc:1511] Processing CreateTablet for tablet 92a59e9527744e8e99f0b0b20b98e729 (DEFAULT_TABLE table=default.Table_86e17c4523fa4812a508156cff380a35 [id=3bef47b1fe5f4403b2b5e615f7baeaf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.621765 12730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a59e9527744e8e99f0b0b20b98e729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.624841 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a59e9527744e8e99f0b0b20b98e729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.630195 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 6a85852c08304e0fbbbcbb2c0d24b27e (DEFAULT_TABLE table=default.Table_019731a69622436792bb7a8512e0cb49 [id=f2cc65dc707940e5a763c0d4b768a072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.631261 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a85852c08304e0fbbbcbb2c0d24b27e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.637228 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 6a85852c08304e0fbbbcbb2c0d24b27e (DEFAULT_TABLE table=default.Table_019731a69622436792bb7a8512e0cb49 [id=f2cc65dc707940e5a763c0d4b768a072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.638355 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a85852c08304e0fbbbcbb2c0d24b27e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.623279 13830 raft_consensus.cc:359] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:56.650310 12729 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1ea70734164c9db379ede12abae1f1 (DEFAULT_TABLE table=default.Table_a95683f794924e4fba011142bea60695 [id=40d0077698944485a23bbc64072df5a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.657542 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1ea70734164c9db379ede12abae1f1 (DEFAULT_TABLE table=default.Table_a95683f794924e4fba011142bea60695 [id=40d0077698944485a23bbc64072df5a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.672888 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 71065b80482442699f4b207cb8f9d40c
I20260926 19:58:56.675248 12556 tablet_service.cc:1558] Processing DeleteTablet for tablet 71065b80482442699f4b207cb8f9d40c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:56 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:56.677660 12393 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5f94ac931883406abfbdd3da405dc80a" } modify_external_catalogs: true
I20260926 19:58:56.677948 13013 tablet_bootstrap.cc:492] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:56.678145 12393 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_5f94ac931883406abfbdd3da405dc80a" } modify_external_catalogs: true
I20260926 19:58:56.678331 13013 ts_tablet_manager.cc:1403] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.079s user 0.009s sys 0.000s
I20260926 19:58:56.680532 13013 raft_consensus.cc:359] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:56.689596 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 92a59e9527744e8e99f0b0b20b98e729 (DEFAULT_TABLE table=default.Table_86e17c4523fa4812a508156cff380a35 [id=3bef47b1fe5f4403b2b5e615f7baeaf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.690721 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a59e9527744e8e99f0b0b20b98e729. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.691383 13013 raft_consensus.cc:385] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:56.691641 13013 raft_consensus.cc:740] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:56.691681 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 6a85852c08304e0fbbbcbb2c0d24b27e (DEFAULT_TABLE table=default.Table_019731a69622436792bb7a8512e0cb49 [id=f2cc65dc707940e5a763c0d4b768a072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.692582 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a85852c08304e0fbbbcbb2c0d24b27e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.692219 13013 consensus_queue.cc:260] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:56.693619 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet d48ee26c0aaa4abc8e08a1a2dadb028d (table default.Table_1e071d1293234cecb72d6a118e8d5f10 [id=21d0c2efe78e438e8fad11d811c463ed]) successfully deleted
I20260926 19:58:56.696797 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 258) took 2549 ms. Trace:
I20260926 19:58:56.697192 13403 rpcz_store.cc:276] 0926 19:58:54.147000 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.168582 (+ 21582us) service_pool.cc:224] Handling call
0926 19:58:56.653767 (+2485185us) ts_tablet_manager.cc:1151] Deleting tablet 6c05c67d89814ca6bc906f4b683523b0
0926 19:58:56.653816 (+ 49us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:56.696741 (+ 42925us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2480620}
I20260926 19:58:56.700278 12904 tablet_service.cc:1511] Processing CreateTablet for tablet fd98f43d7000438da50cc5285a4395ae (DEFAULT_TABLE table=default.Table_a73209f0f4544df897c1e856f3692fe8 [id=813b847fffb449f497ef2bc3ff10186a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.701244 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd98f43d7000438da50cc5285a4395ae. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:56.701953 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c38a5aab5eb4404b783eaa5cc477c31 has no consensus state
I20260926 19:58:56.705670 12895 tablet_service.cc:1558] Processing DeleteTablet for tablet 90b7a9622b0f412b9a88aa84419ad0d7 with delete_type TABLET_DATA_DELETED (replaced by tablet a0c465b503b54e569fbf529da3ac316c) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:56.706341 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 71065b80482442699f4b207cb8f9d40c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:56 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.706892 12729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1ea70734164c9db379ede12abae1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.706823 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 9b1ea70734164c9db379ede12abae1f1 (DEFAULT_TABLE table=default.Table_a95683f794924e4fba011142bea60695 [id=40d0077698944485a23bbc64072df5a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.707726 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1ea70734164c9db379ede12abae1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.709714 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet 71065b80482442699f4b207cb8f9d40c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:56 UTC) from {username='slave'} at 127.0.0.1:33594
W20260926 19:58:56.714357 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 71065b80482442699f4b207cb8f9d40c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71065b80482442699f4b207cb8f9d40c already in progress: creating tablet
I20260926 19:58:56.666288 12891 tablet_service.cc:1511] Processing CreateTablet for tablet de71cbac8eef423c897c8c142fd29b61 (DEFAULT_TABLE table=default.Table_652bcc921b9c4e77943bc28b1c6e2166 [id=a30267dc46484a46be8980dcebeb4b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.715734 12891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de71cbac8eef423c897c8c142fd29b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.717869 12387 catalog_manager.cc:5838] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:56.719995 12387 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:58:56.720192 12556 tablet_service.cc:1511] Processing CreateTablet for tablet fd98f43d7000438da50cc5285a4395ae (DEFAULT_TABLE table=default.Table_a73209f0f4544df897c1e856f3692fe8 [id=813b847fffb449f497ef2bc3ff10186a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.720731 12387 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:56.727094 12737 tablet_service.cc:1511] Processing CreateTablet for tablet fd98f43d7000438da50cc5285a4395ae (DEFAULT_TABLE table=default.Table_a73209f0f4544df897c1e856f3692fe8 [id=813b847fffb449f497ef2bc3ff10186a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.727488 12565 tablet_service.cc:1511] Processing CreateTablet for tablet d9adc53fada24b55bc81737eaaca902f (DEFAULT_TABLE table=default.Table_7201d8013a434923a470d52f3d321e05 [id=e0a34642d6134825ae61f8990d8577f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.728053 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd98f43d7000438da50cc5285a4395ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.735235 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 441b881f3660495cba9b94a8c7464e99 (DEFAULT_TABLE table=default.Table_42e2147189dd490180e130039faf6cdc [id=fedf8b09ae88433aa308e7fbf4c4fac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.736204 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441b881f3660495cba9b94a8c7464e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.738380 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 441b881f3660495cba9b94a8c7464e99 (DEFAULT_TABLE table=default.Table_42e2147189dd490180e130039faf6cdc [id=fedf8b09ae88433aa308e7fbf4c4fac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.742571 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ecc2264951a44e51afb66b0053bc0474"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:56.744726 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ecc2264951a44e51afb66b0053bc0474 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:56.746898 12732 tablet_service.cc:1511] Processing CreateTablet for tablet d9adc53fada24b55bc81737eaaca902f (DEFAULT_TABLE table=default.Table_7201d8013a434923a470d52f3d321e05 [id=e0a34642d6134825ae61f8990d8577f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.746969 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ef93987144814289b97049e7fbc68e1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:56.747857 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9adc53fada24b55bc81737eaaca902f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.747885 13013 ts_tablet_manager.cc:1434] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.069s user 0.005s sys 0.000s
W20260926 19:58:56.748317 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef93987144814289b97049e7fbc68e1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:56.750253 12736 tablet_service.cc:1511] Processing CreateTablet for tablet efb97fbf233143caaf46dc6f5398362b (DEFAULT_TABLE table=default.Table_e572a5b025f943e38097a94a29d245ff [id=8ba529ceaf14407abf603496dcd04d1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.751157 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efb97fbf233143caaf46dc6f5398362b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.751735 12555 tablet_service.cc:1511] Processing CreateTablet for tablet efb97fbf233143caaf46dc6f5398362b (DEFAULT_TABLE table=default.Table_e572a5b025f943e38097a94a29d245ff [id=8ba529ceaf14407abf603496dcd04d1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.769086 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_716c1e6e330044689f2657d0d570f0fd" } new_table_name: "default.Table_c360f50af2554b0984e0b30eb5bbd833" modify_external_catalogs: true
I20260926 19:58:56.776242 12393 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet fc72eb14227d493c9dfbcd852c1d5699
I20260926 19:58:56.807405 12898 tablet_service.cc:1511] Processing CreateTablet for tablet efb97fbf233143caaf46dc6f5398362b (DEFAULT_TABLE table=default.Table_e572a5b025f943e38097a94a29d245ff [id=8ba529ceaf14407abf603496dcd04d1e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:56.808776 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 6c05c67d89814ca6bc906f4b683523b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c05c67d89814ca6bc906f4b683523b0 already in progress: creating tablet
W20260926 19:58:56.809329 12857 consensus_peers.cc:597] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:56.809630 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efb97fbf233143caaf46dc6f5398362b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.810465 12895 tablet_service.cc:1511] Processing CreateTablet for tablet 441b881f3660495cba9b94a8c7464e99 (DEFAULT_TABLE table=default.Table_42e2147189dd490180e130039faf6cdc [id=fedf8b09ae88433aa308e7fbf4c4fac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.811898 12895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441b881f3660495cba9b94a8c7464e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.813651 12897 tablet_service.cc:1511] Processing CreateTablet for tablet d9adc53fada24b55bc81737eaaca902f (DEFAULT_TABLE table=default.Table_7201d8013a434923a470d52f3d321e05 [id=e0a34642d6134825ae61f8990d8577f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:56.814468 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9adc53fada24b55bc81737eaaca902f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.818715 12393 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_36d6c33f2b4f465f9f2b8823a09b3bfc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:56.820149 12393 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36d6c33f2b4f465f9f2b8823a09b3bfc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:56.808866 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:56.832712 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:56.839987 12389 master_service.cc:960] ReplaceTablet: received request to replace tablet a26e6b837f67445c9383599b7e52de7b from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:56.841944 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:58:56.846153 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 260) took 2613 ms. Trace:
I20260926 19:58:56.846480 13403 rpcz_store.cc:276] 0926 19:58:54.232303 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.232469 (+ 166us) service_pool.cc:224] Handling call
0926 19:58:56.809038 (+2576569us) ts_tablet_manager.cc:1151] Deleting tablet 13bb4e7eeef145fc85a5c412add99ce7
0926 19:58:56.845820 (+ 36782us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:56.846086 (+ 266us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2575487}
I20260926 19:58:56.847740 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 261) took 2613 ms. Trace:
W20260926 19:58:56.847868 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 13bb4e7eeef145fc85a5c412add99ce7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 13bb4e7eeef145fc85a5c412add99ce7 already in progress: creating tablet
I20260926 19:58:56.848021 13403 rpcz_store.cc:276] 0926 19:58:54.233954 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.234048 (+ 94us) service_pool.cc:224] Handling call
0926 19:58:56.847436 (+2613388us) ts_tablet_manager.cc:1151] Deleting tablet 2ad76fc959e24545bdfad19f988fce93
0926 19:58:56.847474 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:56.847689 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2612909}
I20260926 19:58:56.849071 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 263) took 1891 ms. Trace:
W20260926 19:58:56.849155 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 2ad76fc959e24545bdfad19f988fce93 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ad76fc959e24545bdfad19f988fce93 already in progress: creating tablet
I20260926 19:58:56.850061 13830 raft_consensus.cc:740] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2490adb6e84090b780f52470e580ab, State: Initialized, Role: FOLLOWER
I20260926 19:58:56.852607 13803 leader_election.cc:290] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:56.853886 13830 consensus_queue.cc:260] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:56.849356 13403 rpcz_store.cc:276] 0926 19:58:54.957988 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:54.959136 (+ 1148us) service_pool.cc:224] Handling call
0926 19:58:56.848804 (+1889668us) ts_tablet_manager.cc:1151] Deleting tablet 0df471a7b6554d4fb5007adb63babe43
0926 19:58:56.848838 (+ 34us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:56.849027 (+ 189us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1873907}
W20260926 19:58:56.866248 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 0df471a7b6554d4fb5007adb63babe43 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0df471a7b6554d4fb5007adb63babe43 already in progress: creating tablet
I20260926 19:58:56.867141 13403 tablet_replica.cc:333] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:56.868284 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "acdb4bd8cd58444cb1d6d4b5610de20d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:56.868822 12759 raft_consensus.cc:3060] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:56.874091 13403 raft_consensus.cc:2243] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:56.874639 13403 raft_consensus.cc:2272] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:56.875466 12759 raft_consensus.cc:2468] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:56.876284 13403 ts_tablet_manager.cc:1916] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:56.878856 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_01a9ebf62e394638933953cc6654b838" } modify_external_catalogs: true
I20260926 19:58:56.844993 12395 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49]). Expected version 1 got 0
I20260926 19:58:56.885329 13830 sys_catalog.cc:565] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:58:56.877518 13886 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:56.889022 13886 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:56.894021 12857 leader_election.cc:304] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:56.899479 13012 tablet_bootstrap.cc:492] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:56.899904 13012 ts_tablet_manager.cc:1403] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.468s user 0.008s sys 0.010s
I20260926 19:58:56.913030 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b1ea70734164c9db379ede12abae1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.916750 13013 tablet_bootstrap.cc:492] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:56.922010 12556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd98f43d7000438da50cc5285a4395ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.922590 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441b881f3660495cba9b94a8c7464e99. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:56.922515 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 71065b80482442699f4b207cb8f9d40c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71065b80482442699f4b207cb8f9d40c already in progress: creating tablet
I20260926 19:58:56.923074 12555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efb97fbf233143caaf46dc6f5398362b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.925709 13012 raft_consensus.cc:359] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.926417 13012 raft_consensus.cc:385] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:56.927851 13012 raft_consensus.cc:740] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:56.928644 13012 consensus_queue.cc:260] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:56.929567 13013 tablet_bootstrap.cc:654] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.929883 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9adc53fada24b55bc81737eaaca902f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:56.935248 13012 ts_tablet_manager.cc:1434] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260926 19:58:56.942915 13012 tablet_bootstrap.cc:492] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:56.941473 12388 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_d5a60275d2d54a5cb84fac0e1f41e6e7 [id=e452740c55c345a9ab7445009c7a71cb] alter complete (version 2)
W20260926 19:58:56.951045 12388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 64) took 1760 ms (client timeout 1000 ms). Trace:
I20260926 19:58:56.956732 13871 consensus_queue.cc:1048] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:56.957469 13803 raft_consensus.cc:2804] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:56.957978 13803 raft_consensus.cc:697] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
W20260926 19:58:56.958449 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 44791us
I20260926 19:58:56.959482 13803 consensus_queue.cc:237] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:56.951373 12388 rpcz_store.cc:269] 0926 19:58:55.191048 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.254324 (+ 63276us) service_pool.cc:224] Handling call
0926 19:58:55.313634 (+ 59310us) spinlock_profiling.cc:243] Waited 57.5 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:56.950983 (+1637349us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:55.697633 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:55.697998 (+ 365us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:55.698018 (+ 20us) write_op.cc:435] Acquired schema lock
0926 19:58:55.698031 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:55.698271 (+ 240us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:55.698309 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:55.698331 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:55.698465 (+ 134us) tablet.cc:701] Row locks acquired
0926 19:58:55.698478 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:55.698581 (+ 103us) write_op.cc:270] Start()
0926 19:58:55.699508 (+ 927us) write_op.cc:276] Timestamp: P: 1790452735698520 usec, L: 0
0926 19:58:55.699526 (+ 18us) op_driver.cc:348] REPLICATION: starting
0926 19:58:55.699978 (+ 452us) log.cc:844] Serialized 356 byte log entry
0926 19:58:55.750098 (+ 50120us) op_driver.cc:464] REPLICATION: finished
0926 19:58:55.750473 (+ 375us) write_op.cc:301] APPLY: starting
0926 19:58:55.750525 (+ 52us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:55.750613 (+ 88us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:55.750706 (+ 93us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:55.750878 (+ 172us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:55.751000 (+ 122us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:55.751049 (+ 49us) write_op.cc:312] APPLY: finished
0926 19:58:55.751212 (+ 163us) log.cc:844] Serialized 23 byte log entry
0926 19:58:55.751355 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:55.751418 (+ 63us) write_op.cc:454] Released schema lock
0926 19:58:55.751448 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:55.769955 (+ 18507us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:55.769979 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8895,"reported_tablets":1,"spinlock_wait_cycles":132362624,"child_traces":[["op",{"apply.queue_time_us":241,"apply.run_cpu_time_us":1152,"apply.run_wall_time_us":1151,"num_ops":1,"prepare.queue_time_us":378538,"prepare.run_cpu_time_us":1919,"prepare.run_wall_time_us":72303,"raft.queue_time_us":6779,"raft.run_cpu_time_us":804,"raft.run_wall_time_us":804,"replication_time_us":50472}]]}
I20260926 19:58:56.964932 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_e70be55e31fe48c8a83266f2d0937cab" } new_table_name: "default.Table_156126e29ad847b1ad7d29c9b69aad51" modify_external_catalogs: true
W20260926 19:58:56.965547 12391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 65) took 755 ms (client timeout 1000 ms). Trace:
W20260926 19:58:56.965884 12391 rpcz_store.cc:269] 0926 19:58:56.209848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.211297 (+ 1449us) service_pool.cc:224] Handling call
0926 19:58:56.945497 (+734200us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:56.661996 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:56.662347 (+ 351us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:56.662369 (+ 22us) write_op.cc:435] Acquired schema lock
0926 19:58:56.662382 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:56.662604 (+ 222us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:56.662637 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:56.662658 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:56.662786 (+ 128us) tablet.cc:701] Row locks acquired
0926 19:58:56.662799 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:56.662894 (+ 95us) write_op.cc:270] Start()
0926 19:58:56.662965 (+ 71us) write_op.cc:276] Timestamp: P: 1790452736662838 usec, L: 0
0926 19:58:56.662980 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:56.663405 (+ 425us) log.cc:844] Serialized 536 byte log entry
0926 19:58:56.693157 (+ 29752us) op_driver.cc:464] REPLICATION: finished
0926 19:58:56.694166 (+ 1009us) write_op.cc:301] APPLY: starting
0926 19:58:56.694219 (+ 53us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:56.694302 (+ 83us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:56.694390 (+ 88us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:56.694569 (+ 179us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:56.694725 (+ 156us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:56.694789 (+ 64us) write_op.cc:312] APPLY: finished
0926 19:58:56.695004 (+ 215us) log.cc:844] Serialized 23 byte log entry
0926 19:58:56.695195 (+ 191us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:56.695282 (+ 87us) write_op.cc:454] Released schema lock
0926 19:58:56.695334 (+ 52us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:56.695612 (+ 278us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:56.695635 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":862,"apply.run_cpu_time_us":1416,"apply.run_wall_time_us":1416,"num_ops":1,"prepare.queue_time_us":83502,"prepare.run_cpu_time_us":1627,"prepare.run_wall_time_us":13286,"replication_time_us":30072}]]}
I20260926 19:58:56.973207 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet c5b734f0d33342d3be8b1827bcd5dd14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:56.974706 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet c5b734f0d33342d3be8b1827bcd5dd14 because the tablet was not found. No further retry: Not found: Tablet not found: c5b734f0d33342d3be8b1827bcd5dd14
I20260926 19:58:56.975033 13012 tablet_bootstrap.cc:654] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:56.976579 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet cd86f01d82d041f4852f83f2547e697f with delete_type TABLET_DATA_DELETED (replaced by tablet 3488f0076b0f4ef19f7cf23a93cd20e3) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.977254 13569 tablet_replica.cc:333] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:56.977955 13569 raft_consensus.cc:2243] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:56.978588 13569 pending_rounds.cc:70] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Trying to abort 1 pending ops.
I20260926 19:58:56.978786 13569 pending_rounds.cc:77] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694405965103104 op_type: NO_OP noop_request { }
I20260926 19:58:56.979133 13569 raft_consensus.cc:2889] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:56.979458 13569 raft_consensus.cc:2272] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:56.982739 13569 ts_tablet_manager.cc:1916] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:56.988147 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.991431 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet 66c8916d88dd48998eab514bd3c538b9 with delete_type TABLET_DATA_DELETED (replaced by tablet 700d05149a934d5e8bec9cc41873ad3e) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.992777 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d with delete_type TABLET_DATA_DELETED (replaced by tablet 7645307e0999482bbe3329c8d61cc567) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.994801 12729 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:44 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:56.995669 12387 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_7a4372f262834b68bade3baafd4658b7" } modify_external_catalogs: true
I20260926 19:58:56.998215 12565 tablet_service.cc:1558] Processing DeleteTablet for tablet d48ee26c0aaa4abc8e08a1a2dadb028d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:46 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:56.999235 12392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 54) took 1906 ms (client timeout 1000 ms). Trace:
W20260926 19:58:56.999737 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet d48ee26c0aaa4abc8e08a1a2dadb028d because the tablet was not found. No further retry: Not found: Tablet not found: d48ee26c0aaa4abc8e08a1a2dadb028d
I20260926 19:58:56.990166 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_d39ede314be3449d96bbf00b8c9be90b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:57.003688 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d39ede314be3449d96bbf00b8c9be90b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:57.010993 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f268b4a1f7423f917731c6b62aab3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:57.011494 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 with delete_type TABLET_DATA_DELETED (replaced by tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:57.012127 12390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 58) took 1603 ms (client timeout 1000 ms). Trace:
W20260926 19:58:57.012606 12390 rpcz_store.cc:269] 0926 19:58:55.408722 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.410969 (+ 2247us) service_pool.cc:224] Handling call
0926 19:58:57.012050 (+1601081us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44964,"reported_tablets":21}
W20260926 19:58:57.015586 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 22f268b4a1f7423f917731c6b62aab3e because the tablet was not found. No further retry: Not found: Tablet not found: 22f268b4a1f7423f917731c6b62aab3e
I20260926 19:58:57.015962 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d with delete_type TABLET_DATA_DELETED (replaced by tablet 7645307e0999482bbe3329c8d61cc567) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:57.016207 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet 5930402db43740b4b291c6c238ecf52c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:57.016695 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet b32f8b0d480547a58ba5c6fed6483a74 because the tablet was not found. No further retry: Not found: Tablet not found: b32f8b0d480547a58ba5c6fed6483a74
I20260926 19:58:57.016793 12561 tablet_service.cc:1558] Processing DeleteTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:57.017086 13203 tablet_replica.cc:333] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
W20260926 19:58:57.017539 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet acdb4bd8cd58444cb1d6d4b5610de20d because the tablet was not found. No further retry: Not found: Tablet not found: acdb4bd8cd58444cb1d6d4b5610de20d
I20260926 19:58:57.018031 13203 raft_consensus.cc:2243] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:57.018498 13203 raft_consensus.cc:2272] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:57.020136 13203 ts_tablet_manager.cc:1916] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:58:56.999536 12392 rpcz_store.cc:269] 0926 19:58:55.092677 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.159912 (+ 67235us) service_pool.cc:224] Handling call
0926 19:58:55.319524 (+159612us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:56.999172 (+1679648us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:56.422672 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:56.423044 (+ 372us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:56.423070 (+ 26us) write_op.cc:435] Acquired schema lock
0926 19:58:56.423083 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:56.423326 (+ 243us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:56.423364 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:56.423389 (+ 25us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:56.423541 (+ 152us) tablet.cc:701] Row locks acquired
0926 19:58:56.423555 (+ 14us) write_op.cc:260] PREPARE: finished
0926 19:58:56.423657 (+ 102us) write_op.cc:270] Start()
0926 19:58:56.423735 (+ 78us) write_op.cc:276] Timestamp: P: 1790452736423611 usec, L: 0
0926 19:58:56.423751 (+ 16us) op_driver.cc:348] REPLICATION: starting
0926 19:58:56.424182 (+ 431us) log.cc:844] Serialized 536 byte log entry
0926 19:58:56.519780 (+ 95598us) op_driver.cc:464] REPLICATION: finished
0926 19:58:56.520503 (+ 723us) write_op.cc:301] APPLY: starting
0926 19:58:56.520553 (+ 50us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:56.520647 (+ 94us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:56.520731 (+ 84us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:56.520897 (+ 166us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:56.521002 (+ 105us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:56.521055 (+ 53us) write_op.cc:312] APPLY: finished
0926 19:58:56.521183 (+ 128us) log.cc:844] Serialized 23 byte log entry
0926 19:58:56.521288 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:56.521332 (+ 44us) write_op.cc:454] Released schema lock
0926 19:58:56.521361 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:56.521558 (+ 197us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:56.521582 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9455,"reported_tablets":14,"spinlock_wait_cycles":193090816,"child_traces":[["op",{"apply.queue_time_us":599,"apply.run_cpu_time_us":1005,"apply.run_wall_time_us":1005,"num_ops":1,"prepare.queue_time_us":61083,"prepare.run_cpu_time_us":1698,"prepare.run_wall_time_us":2559,"replication_time_us":95952}]]}
I20260926 19:58:57.023434 13856 raft_consensus.cc:493] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.023867 13856 raft_consensus.cc:515] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:57.025729 13856 leader_election.cc:290] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.028729 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f666e1a8d214172bf5b752e9ec374bd" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:57.030033 12519 leader_election.cc:343] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:57.038939 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43a92aa343944eab9c937aeeac953649 has no consensus state
I20260926 19:58:57.039595 12389 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet a26e6b837f67445c9383599b7e52de7b
W20260926 19:58:57.040246 12731 consensus_meta.cc:287] T 5d9e9e5038774f6cb35de166f1fd78d4 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.520s user 0.008s sys 0.000s
I20260926 19:58:57.041394 12389 catalog_manager.cc:6934] ReplaceTablet: tablet a26e6b837f67445c9383599b7e52de7b deleted and replaced by tablet 57643cb928554e70a22c4b213411e0b2
I20260926 19:58:57.042429 12387 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_7a4372f262834b68bade3baafd4658b7" } modify_external_catalogs: true
I20260926 19:58:57.042559 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f666e1a8d214172bf5b752e9ec374bd" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:57.043179 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_2aae6d0b16cf43b8ade8e77279c797b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:57.044116 12521 leader_election.cc:343] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:57.044472 12521 leader_election.cc:304] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
W20260926 19:58:57.044710 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2aae6d0b16cf43b8ade8e77279c797b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:57.045351 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_01a9ebf62e394638933953cc6654b838" } modify_external_catalogs: true
I20260926 19:58:57.045841 13856 raft_consensus.cc:2749] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:57.049232 13888 raft_consensus.cc:493] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.049685 13888 raft_consensus.cc:515] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.050364 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet a26e6b837f67445c9383599b7e52de7b with delete_type TABLET_DATA_DELETED (replaced by tablet 57643cb928554e70a22c4b213411e0b2) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:57.051664 13856 raft_consensus.cc:493] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.052093 13888 leader_election.cc:290] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.052062 13856 raft_consensus.cc:515] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.054594 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:57.055246 13012 tablet_bootstrap.cc:492] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:57.055200 12918 raft_consensus.cc:2468] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
I20260926 19:58:57.055658 13012 ts_tablet_manager.cc:1403] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.113s user 0.003s sys 0.005s
I20260926 19:58:57.057184 12521 leader_election.cc:304] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:57.058043 13856 leader_election.cc:290] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.058624 13856 raft_consensus.cc:2804] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 19:58:57.059227 13856 raft_consensus.cc:493] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:57.060009 13856 raft_consensus.cc:3060] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:57.062238 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:57.062839 12759 raft_consensus.cc:2468] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 1.
W20260926 19:58:57.057147 13092 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.662s user 0.004s sys 0.032s
W20260926 19:58:57.065055 13092 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.670s user 0.005s sys 0.033s
I20260926 19:58:57.062225 13012 raft_consensus.cc:359] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
W20260926 19:58:57.068789 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 109688us
I20260926 19:58:57.069959 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_a9722bfab4444bfabadaa2118c2da3b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:57.071209 13012 raft_consensus.cc:385] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.071480 13012 raft_consensus.cc:740] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
W20260926 19:58:57.072160 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9722bfab4444bfabadaa2118c2da3b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:57.078114 13856 raft_consensus.cc:515] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.076593 13012 consensus_queue.cc:260] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.080765 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet a26e6b837f67445c9383599b7e52de7b with delete_type TABLET_DATA_DELETED (replaced by tablet 57643cb928554e70a22c4b213411e0b2) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:57.081259 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:57.081833 12759 raft_consensus.cc:3060] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:57.082384 13203 ts_tablet_manager.cc:1929] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:57.082711 13203 log.cc:1199] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/5930402db43740b4b291c6c238ecf52c
I20260926 19:58:57.083298 13203 ts_tablet_manager.cc:1950] T 5930402db43740b4b291c6c238ecf52c P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:57.084671 13012 ts_tablet_manager.cc:1434] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.029s user 0.007s sys 0.001s
I20260926 19:58:57.089032 12759 raft_consensus.cc:2468] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 2.
I20260926 19:58:57.090380 12519 leader_election.cc:304] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:57.096390 13092 tablet_bootstrap.cc:492] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:57.098697 13856 leader_election.cc:290] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.099593 13012 tablet_bootstrap.cc:492] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:57.107276 13856 raft_consensus.cc:2804] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:58:57.108879 13856 raft_consensus.cc:697] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [term 2 LEADER]: Becoming Leader. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Running, Role: LEADER
I20260926 19:58:57.112345 13092 ts_tablet_manager.cc:1403] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.829s user 0.013s sys 0.035s
I20260926 19:58:57.118705 13856 consensus_queue.cc:237] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:57.119979 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 3df392d277a447aabf4320a7d9a43880 (table default.Table_71b660a4d8c04aae800e0747908f556e [id=f4a644d228fe4e47985e6142a522275d], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3df392d277a447aabf4320a7d9a43880 consensus state has no leader
I20260926 19:58:57.114893 12891 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:44 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.122792 12895 tablet_service.cc:1558] Processing DeleteTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.125133 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet cd86f01d82d041f4852f83f2547e697f with delete_type TABLET_DATA_DELETED (replaced by tablet 3488f0076b0f4ef19f7cf23a93cd20e3) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.126395 12918 raft_consensus.cc:3060] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:57.129961 12555 tablet_service.cc:1558] Processing DeleteTablet for tablet 59f7dc92330d400f93f427f3f79ad826 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d153e60ae5d486cb91c2cc623b99985) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:57.130807 13203 tablet_replica.cc:333] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:57.131655 13203 raft_consensus.cc:2243] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:57.134508 13203 raft_consensus.cc:2272] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:57.135282 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d with delete_type TABLET_DATA_DELETED (replaced by tablet 7645307e0999482bbe3329c8d61cc567) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.136289 13203 ts_tablet_manager.cc:1916] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:57.137624 13092 raft_consensus.cc:359] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.131791 13013 tablet_bootstrap.cc:492] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:57.140035 13013 ts_tablet_manager.cc:1403] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.223s user 0.006s sys 0.018s
I20260926 19:58:57.147238 13012 tablet_bootstrap.cc:654] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:57.147591 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 57299.8us
I20260926 19:58:57.149626 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2d4d45fdeb4802ac261b6c8de91c3c" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:57.150455 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b2d4d45fdeb4802ac261b6c8de91c3c" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:57.138651 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet a26e6b837f67445c9383599b7e52de7b with delete_type TABLET_DATA_DELETED (replaced by tablet 57643cb928554e70a22c4b213411e0b2) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.154752 13092 raft_consensus.cc:385] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.155068 13092 raft_consensus.cc:740] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.156774 13092 consensus_queue.cc:260] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.164407 13092 ts_tablet_manager.cc:1434] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.041s user 0.003s sys 0.001s
W20260926 19:58:57.144663 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 87541.7us
I20260926 19:58:57.169430 12393 master_service.cc:960] ReplaceTablet: received request to replace tablet 9411d3bca81641a4b3dfaffb2dc2bb51 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:57.173261 13092 tablet_bootstrap.cc:492] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:57.174748 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d1955b12dd47899b8a226fe0e980ae" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:57.175261 12920 raft_consensus.cc:3060] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:58:57.179598 13012 tablet_bootstrap.cc:492] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:57.180081 13012 ts_tablet_manager.cc:1403] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.095s user 0.008s sys 0.002s
I20260926 19:58:57.182296 12857 leader_election.cc:304] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:57.182484 12920 raft_consensus.cc:2468] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6e9d6439fd54b28925a68a6145d3dee in term 2.
I20260926 19:58:57.187898 12395 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:58:57.182415 13012 raft_consensus.cc:359] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.190071 13012 raft_consensus.cc:385] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.193620 13803 raft_consensus.cc:2749] T 9b2d4d45fdeb4802ac261b6c8de91c3c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:57.193885 13012 raft_consensus.cc:740] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.194478 13012 consensus_queue.cc:260] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.205351 12970 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
W20260926 19:58:57.207084 12381 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.254:33143, user_credentials={real_user=slave}} blocked reactor thread for 68335.2us
I20260926 19:58:57.207741 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 5930402db43740b4b291c6c238ecf52c (table default.Table_a195c0af637546dc947815ce7a384b8b [id=56f80b8cc8eb47358c033dc894be8645]) successfully deleted
W20260926 19:58:57.208946 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 89c8cbb097a644218e0ca033a6fdb4a2 because the tablet was not found. No further retry: Not found: Tablet not found: 89c8cbb097a644218e0ca033a6fdb4a2
W20260926 19:58:57.209540 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet a26e6b837f67445c9383599b7e52de7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a26e6b837f67445c9383599b7e52de7b already in progress: creating tablet
I20260926 19:58:57.212091 13012 ts_tablet_manager.cc:1434] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.032s user 0.001s sys 0.004s
I20260926 19:58:57.214614 13803 raft_consensus.cc:493] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.215015 13803 raft_consensus.cc:515] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.221040 13203 ts_tablet_manager.cc:1929] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:57.221576 13803 leader_election.cc:290] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.225221 13012 tablet_bootstrap.cc:492] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:57.230104 12395 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:58:57.231326 13092 tablet_bootstrap.cc:654] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:57.231705 12695 consensus_peers.cc:597] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 06c5ef9e1bca427fb889cbb17f77f999. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:57.232251 13013 raft_consensus.cc:359] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:57.232883 13013 raft_consensus.cc:385] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.233119 13013 raft_consensus.cc:740] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.234370 13856 raft_consensus.cc:493] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.234860 13856 raft_consensus.cc:515] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:57.237298 13856 leader_election.cc:290] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.238029 13874 raft_consensus.cc:493] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.238462 13874 raft_consensus.cc:515] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.240288 13874 leader_election.cc:290] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:57.245690 12391 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_acdb314f8df24fea96383e9e16a5efe6" } new_table_name: "default.Table_2246cb86fb8e49de90ee0350d6287a8c" modify_external_catalogs: true
I20260926 19:58:57.242708 13203 log.cc:1199] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/59f7dc92330d400f93f427f3f79ad826
I20260926 19:58:57.241788 13013 consensus_queue.cc:260] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:57.254113 13203 ts_tablet_manager.cc:1950] T 59f7dc92330d400f93f427f3f79ad826 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:57.255196 13013 ts_tablet_manager.cc:1434] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.115s user 0.005s sys 0.000s
I20260926 19:58:57.261133 13092 tablet_bootstrap.cc:492] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:57.261610 13092 ts_tablet_manager.cc:1403] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.089s user 0.007s sys 0.003s
I20260926 19:58:57.263916 13092 raft_consensus.cc:359] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.267150 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89c8cbb097a644218e0ca033a6fdb4a2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:57.267711 12919 raft_consensus.cc:2468] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:57.264655 13092 raft_consensus.cc:385] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.272600 13092 raft_consensus.cc:740] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.273335 13092 consensus_queue.cc:260] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.275094 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb035b50e8f445ba492e67796085123" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:57.277009 12521 leader_election.cc:343] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:57.281366 13092 ts_tablet_manager.cc:1434] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
I20260926 19:58:57.292136 13092 tablet_bootstrap.cc:492] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:57.292923 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f268b4a1f7423f917731c6b62aab3e" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:57.293578 12759 raft_consensus.cc:2468] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
W20260926 19:58:57.296461 12381 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.254:33143, user_credentials={real_user=slave}} blocked reactor thread for 81949.8us
I20260926 19:58:57.299587 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f268b4a1f7423f917731c6b62aab3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.304479 13092 tablet_bootstrap.cc:654] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:57.310156 12695 leader_election.cc:343] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 89c8cbb097a644218e0ca033a6fdb4a2
I20260926 19:58:57.312721 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:58:57.313550 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 6f666e1a8d214172bf5b752e9ec374bd (table default.Table_c7d455485dd14d60806f7a48b45ce98b [id=453a40bc60fd4eeaa2ffc14d887320ec]). Expected version 1 got 0
I20260926 19:58:57.318799 13013 tablet_bootstrap.cc:492] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:57.324702 13013 tablet_bootstrap.cc:654] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.325583 13092 tablet_bootstrap.cc:492] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:57.326040 13092 ts_tablet_manager.cc:1403] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.004s
I20260926 19:58:57.328382 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb035b50e8f445ba492e67796085123" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:57.334931 13092 raft_consensus.cc:359] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:57.352918 12519 leader_election.cc:343] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:57.353327 12519 leader_election.cc:304] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
W20260926 19:58:57.350288 12858 leader_election.cc:343] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 22f268b4a1f7423f917731c6b62aab3e
I20260926 19:58:57.367470 13092 raft_consensus.cc:385] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.369973 13092 raft_consensus.cc:740] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.371516 13092 consensus_queue.cc:260] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.374387 13012 tablet_bootstrap.cc:654] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.376366 13092 ts_tablet_manager.cc:1434] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.050s user 0.004s sys 0.001s
I20260926 19:58:57.390151 13092 tablet_bootstrap.cc:492] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
W20260926 19:58:57.400388 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 105685us
I20260926 19:58:57.404078 12696 leader_election.cc:304] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:57.404893 13840 raft_consensus.cc:2804] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:57.405234 13840 raft_consensus.cc:493] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:57.405571 13840 raft_consensus.cc:3060] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:57.406088 13092 tablet_bootstrap.cc:654] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.419286 13013 tablet_bootstrap.cc:492] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:57.419812 13013 ts_tablet_manager.cc:1403] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.101s user 0.002s sys 0.008s
I20260926 19:58:57.328094 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b840208ddba74ea084e0e59efccb56f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:57.422376 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b840208ddba74ea084e0e59efccb56f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:57.427798 13856 raft_consensus.cc:2749] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:57.428551 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 73403.2us
W20260926 19:58:57.428802 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 75914.9us
I20260926 19:58:57.436724 13569 ts_tablet_manager.cc:1929] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:57.438382 12392 catalog_manager.cc:5838] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:58:57.440150 12392 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:58:57.454913 13092 tablet_bootstrap.cc:492] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:57.455312 13092 ts_tablet_manager.cc:1403] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.005s
I20260926 19:58:57.457587 13092 raft_consensus.cc:359] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.458176 13092 raft_consensus.cc:385] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.458426 13092 raft_consensus.cc:740] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.459025 13092 consensus_queue.cc:260] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.482777 13013 raft_consensus.cc:359] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:57.483577 13013 raft_consensus.cc:385] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.484308 13013 raft_consensus.cc:740] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.484999 13013 consensus_queue.cc:260] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:57.485137 13092 ts_tablet_manager.cc:1434] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.030s user 0.006s sys 0.001s
I20260926 19:58:57.486259 13403 ts_tablet_manager.cc:1929] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:57.486508 13092 tablet_bootstrap.cc:492] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:57.486671 13403 log.cc:1199] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/16897e7489e9445ba5389a4f3507a739
I20260926 19:58:57.487437 13403 ts_tablet_manager.cc:1950] T 16897e7489e9445ba5389a4f3507a739 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:57.489346 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 264) took 2483 ms. Trace:
I20260926 19:58:57.370718 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 59f7dc92330d400f93f427f3f79ad826 (table default.Table_85695893a5b24687959b83a8360d1443 [id=185449ae1c9744eba4df8bd0f9704abb]) successfully deleted
W20260926 19:58:57.490367 12381 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 119661us
I20260926 19:58:57.497298 12393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 9411d3bca81641a4b3dfaffb2dc2bb51
I20260926 19:58:57.498600 12393 catalog_manager.cc:6934] ReplaceTablet: tablet 9411d3bca81641a4b3dfaffb2dc2bb51 deleted and replaced by tablet ddfc561dfbe9481bb327f3a2b7332e31
I20260926 19:58:57.522696 13569 log.cc:1199] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/cd86f01d82d041f4852f83f2547e697f
I20260926 19:58:57.526000 13569 ts_tablet_manager.cc:1950] T cd86f01d82d041f4852f83f2547e697f P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
W20260926 19:58:57.463373 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 59380.5us
I20260926 19:58:57.524513 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 with delete_type TABLET_DATA_DELETED (replaced by tablet ddfc561dfbe9481bb327f3a2b7332e31) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:57.536808 13012 tablet_bootstrap.cc:492] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:57.534065 13915 raft_consensus.cc:493] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.547317 13012 ts_tablet_manager.cc:1403] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.322s user 0.011s sys 0.000s
I20260926 19:58:57.547209 13915 raft_consensus.cc:515] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.549136 13915 leader_election.cc:290] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:57.412984 13840 raft_consensus.cc:515] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.550859 13840 leader_election.cc:290] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:57.561107 13874 consensus_queue.cc:1048] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:57.561686 13840 consensus_queue.cc:1048] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:57.567679 12390 master_service.cc:960] ReplaceTablet: received request to replace tablet e9c7b63db44742108dc132df5089e0d9 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:57.561213 13012 raft_consensus.cc:359] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.576014 13012 raft_consensus.cc:385] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.576365 13012 raft_consensus.cc:740] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.577546 13012 consensus_queue.cc:260] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.579959 13012 ts_tablet_manager.cc:1434] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260926 19:58:57.582913 13012 tablet_bootstrap.cc:492] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:57.589643 13840 consensus_queue.cc:1048] T 791b4ef9be004d7ead26591c50c6f405 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:57.601183 13012 tablet_bootstrap.cc:654] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.633597 13012 tablet_bootstrap.cc:492] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:57.634061 13012 ts_tablet_manager.cc:1403] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.051s user 0.016s sys 0.005s
I20260926 19:58:57.636369 13012 raft_consensus.cc:359] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.639508 13012 raft_consensus.cc:385] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.639860 13012 raft_consensus.cc:740] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.640115 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet cd86f01d82d041f4852f83f2547e697f (table default.Table_87cff927dd5d4f0d9d947b7a5629728c [id=6a77f8956ec6491d915b40237d54a548]) successfully deleted
I20260926 19:58:57.640508 13012 consensus_queue.cc:260] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.642776 13012 ts_tablet_manager.cc:1434] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
I20260926 19:58:57.646749 13012 tablet_bootstrap.cc:492] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:57.509992 12556 tablet_service.cc:1558] Processing DeleteTablet for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 with delete_type TABLET_DATA_DELETED (replaced by tablet ddfc561dfbe9481bb327f3a2b7332e31) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:57.550863 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0583424ef64d9ea3beeb1c01ceafba" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:57.656356 13012 tablet_bootstrap.cc:654] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.657567 13013 ts_tablet_manager.cc:1434] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.237s user 0.006s sys 0.000s
I20260926 19:58:57.503013 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 with delete_type TABLET_DATA_DELETED (replaced by tablet ddfc561dfbe9481bb327f3a2b7332e31) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.516288 13092 tablet_bootstrap.cc:654] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.661296 13403 rpcz_store.cc:276] 0926 19:58:55.005477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.014113 (+ 8636us) service_pool.cc:224] Handling call
0926 19:58:56.866988 (+1852875us) ts_tablet_manager.cc:1151] Deleting tablet 16897e7489e9445ba5389a4f3507a739
0926 19:58:56.867027 (+ 39us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:57.187018 (+319991us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:57.486132 (+299114us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:57.489297 (+ 3165us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1787439}
I20260926 19:58:57.662842 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 16897e7489e9445ba5389a4f3507a739 (table default.Table_646a921cdd5c47d988e91966a807b79b [id=948f2bdffbc84b0ebf0f88e273f0a1fe]) successfully deleted
I20260926 19:58:57.541043 12857 leader_election.cc:304] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:57.663241 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet cfea8dbdd1ba49a5bd9d17652eb271c3
I20260926 19:58:57.663710 13403 tablet_replica.cc:333] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
W20260926 19:58:57.663851 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 122881us
I20260926 19:58:57.665373 12556 tablet_service.cc:1558] Processing DeleteTablet for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:57.665599 13803 raft_consensus.cc:2804] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:57.665917 13803 raft_consensus.cc:493] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:57.666155 12897 tablet_service.cc:1558] Processing DeleteTablet for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.558903 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89c8cbb097a644218e0ca033a6fdb4a2" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:57.666883 12920 raft_consensus.cc:3060] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:57.554901 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0583424ef64d9ea3beeb1c01ceafba" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:57.666209 13803 raft_consensus.cc:3060] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:57.573545 12895 tablet_service.cc:1558] Processing DeleteTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 with delete_type TABLET_DATA_DELETED (replaced by tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.669952 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:38310
W20260926 19:58:57.671226 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 734c7713a06449c190105c1478b62251 consensus state has no leader
I20260926 19:58:57.672475 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 54553d20d48148ae88fa4cc2ce021779
I20260926 19:58:57.676072 12895 tablet_service.cc:1558] Processing DeleteTablet for tablet 54553d20d48148ae88fa4cc2ce021779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:57.680547 13803 raft_consensus.cc:515] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.681353 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_d148763f180a4d4aa0b91f6577410475"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:57.684454 12556 tablet_service.cc:1558] Processing DeleteTablet for tablet 54553d20d48148ae88fa4cc2ce021779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:57.685542 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d148763f180a4d4aa0b91f6577410475 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:57.686877 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet 54553d20d48148ae88fa4cc2ce021779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:57.688369 12920 raft_consensus.cc:2468] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:57.690003 13803 leader_election.cc:290] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.694355 13092 tablet_bootstrap.cc:492] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:57.694770 13092 ts_tablet_manager.cc:1403] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.208s user 0.009s sys 0.008s
I20260926 19:58:57.696979 13092 raft_consensus.cc:359] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.697598 13092 raft_consensus.cc:385] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.697923 13092 raft_consensus.cc:740] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.702636 13092 consensus_queue.cc:260] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.707798 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f25b2e9ee0e441e9ab4fe05366a52e2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:57.709309 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f25b2e9ee0e441e9ab4fe05366a52e2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:57.732132 13012 tablet_bootstrap.cc:492] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:57.735001 13012 ts_tablet_manager.cc:1403] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.009s
I20260926 19:58:57.740655 13092 ts_tablet_manager.cc:1434] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.046s user 0.004s sys 0.002s
I20260926 19:58:57.744141 13092 tablet_bootstrap.cc:492] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:57.742815 13012 raft_consensus.cc:359] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.749892 13012 raft_consensus.cc:385] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.753940 13012 raft_consensus.cc:740] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.755403 13901 raft_consensus.cc:493] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.755849 13901 raft_consensus.cc:515] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:57.766109 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7d0435f3564cb38b6cbca713009c2e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:57.767486 12521 leader_election.cc:343] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:57.769097 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7d0435f3564cb38b6cbca713009c2e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:57.770711 12695 leader_election.cc:343] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 89c8cbb097a644218e0ca033a6fdb4a2
W20260926 19:58:57.771459 12519 leader_election.cc:343] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:57.772092 12519 leader_election.cc:304] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:57.766750 13901 leader_election.cc:290] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.772912 13856 raft_consensus.cc:2749] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:57.773830 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9411d3bca81641a4b3dfaffb2dc2bb51 already in progress: creating tablet
W20260926 19:58:57.774639 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfea8dbdd1ba49a5bd9d17652eb271c3 already in progress: creating tablet
I20260926 19:58:57.774425 13012 consensus_queue.cc:260] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:57.775207 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 54553d20d48148ae88fa4cc2ce021779 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54553d20d48148ae88fa4cc2ce021779 already in progress: creating tablet
W20260926 19:58:57.777909 12858 leader_election.cc:343] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 22f268b4a1f7423f917731c6b62aab3e
I20260926 19:58:57.639572 13569 tablet_replica.cc:333] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:57.778873 13569 raft_consensus.cc:2243] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:57.786573 13013 tablet_bootstrap.cc:492] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:57.790714 13915 raft_consensus.cc:493] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.791220 13012 ts_tablet_manager.cc:1434] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.056s user 0.005s sys 0.001s
I20260926 19:58:57.791756 13569 raft_consensus.cc:2272] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:57.791291 13915 raft_consensus.cc:515] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.793493 13569 ts_tablet_manager.cc:1916] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:57.794893 13092 tablet_bootstrap.cc:654] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.794301 13915 leader_election.cc:290] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:57.803723 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:57.805711 12592 raft_consensus.cc:2468] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:57.807207 12695 leader_election.cc:304] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:57.807900 13915 raft_consensus.cc:2804] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:57.808277 13915 raft_consensus.cc:493] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:57.808785 13915 raft_consensus.cc:3060] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:57.664395 13403 raft_consensus.cc:2243] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:57.832844 13915 raft_consensus.cc:515] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.836826 12591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:57.837528 12591 raft_consensus.cc:3060] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:57.859650 13915 leader_election.cc:290] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:57.862787 13403 raft_consensus.cc:2272] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:57.864573 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b8021096ec5e4b29ada2934d6401a48d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:57.866128 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8021096ec5e4b29ada2934d6401a48d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:57.869230 13901 raft_consensus.cc:493] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.869740 13901 raft_consensus.cc:515] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:57.871013 13092 tablet_bootstrap.cc:492] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:57.871407 13092 ts_tablet_manager.cc:1403] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.130s user 0.008s sys 0.001s
I20260926 19:58:57.871625 13901 leader_election.cc:290] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.873592 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fd4c88f9304f19883c3e5da8f937b5" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:57.874387 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fd4c88f9304f19883c3e5da8f937b5" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:57.875819 13092 raft_consensus.cc:359] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.876492 13092 raft_consensus.cc:385] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.876729 13092 raft_consensus.cc:740] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
W20260926 19:58:57.878263 12521 leader_election.cc:343] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:57.879473 13012 tablet_bootstrap.cc:492] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:57.879777 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c7d455485dd14d60806f7a48b45ce98b" } modify_external_catalogs: true
W20260926 19:58:57.880106 12519 leader_election.cc:343] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:57.880299 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_c7d455485dd14d60806f7a48b45ce98b" } modify_external_catalogs: true
I20260926 19:58:57.880466 12519 leader_election.cc:304] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:57.881373 13901 raft_consensus.cc:2749] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:57.887444 13012 tablet_bootstrap.cc:654] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.896011 13092 consensus_queue.cc:260] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:57.907142 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f268b4a1f7423f917731c6b62aab3e" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:57.907346 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:57.907691 12760 raft_consensus.cc:3060] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:57.907917 12919 raft_consensus.cc:2468] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:57.914304 12760 raft_consensus.cc:2468] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:57.915840 12591 raft_consensus.cc:2468] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:57.920630 13092 ts_tablet_manager.cc:1434] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.049s user 0.006s sys 0.001s
I20260926 19:58:57.921591 13092 tablet_bootstrap.cc:492] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:57.922418 13403 ts_tablet_manager.cc:1916] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:57.927102 12696 leader_election.cc:304] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:57.927774 13915 raft_consensus.cc:2749] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:57.934334 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_2c9b200af3d54549b7c8184d0b0a3d44" } new_table_name: "default.Table_fdb8ab2956a44ea4af05d57738e9abcd" modify_external_catalogs: true
W20260926 19:58:57.934856 12391 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 74fe405570f8474fb7a1f841a783a90a consensus state has no leader
I20260926 19:58:57.935245 13012 tablet_bootstrap.cc:492] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:57.935631 13012 ts_tablet_manager.cc:1403] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.056s user 0.004s sys 0.005s
I20260926 19:58:57.935691 12391 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_4af5dbf295ac46afa888532ae4a8777f" } modify_external_catalogs: true
I20260926 19:58:57.937852 13012 raft_consensus.cc:359] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.941833 13012 raft_consensus.cc:385] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:57.942106 13012 raft_consensus.cc:740] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:57.942754 13013 tablet_bootstrap.cc:654] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.942858 12391 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_4af5dbf295ac46afa888532ae4a8777f" } modify_external_catalogs: true
I20260926 19:58:57.942714 13012 consensus_queue.cc:260] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.946468 12760 raft_consensus.cc:3060] T 1d0583424ef64d9ea3beeb1c01ceafba P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:57.948944 13915 raft_consensus.cc:493] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.949438 13915 raft_consensus.cc:515] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.949668 12696 leader_election.cc:304] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:57.951485 13840 raft_consensus.cc:2804] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:57.951674 13915 leader_election.cc:290] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:57.951920 13840 raft_consensus.cc:697] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:57.953128 13840 consensus_queue.cc:237] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.959127 13012 ts_tablet_manager.cc:1434] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260926 19:58:57.968859 13569 ts_tablet_manager.cc:1929] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:57.969254 13569 log.cc:1199] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/d48ee26c0aaa4abc8e08a1a2dadb028d
I20260926 19:58:57.972108 13012 tablet_bootstrap.cc:492] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:57.975265 13092 tablet_bootstrap.cc:654] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:57.978621 13569 ts_tablet_manager.cc:1950] T d48ee26c0aaa4abc8e08a1a2dadb028d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:57.986392 13803 raft_consensus.cc:493] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:57.986879 13803 raft_consensus.cc:515] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:57.988832 13803 leader_election.cc:290] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:57.991971 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 299) took 1004 ms. Trace:
I20260926 19:58:57.992362 13569 rpcz_store.cc:276] 0926 19:58:56.987877 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.988008 (+ 131us) service_pool.cc:224] Handling call
0926 19:58:57.639416 (+651408us) ts_tablet_manager.cc:1151] Deleting tablet d48ee26c0aaa4abc8e08a1a2dadb028d
0926 19:58:57.639456 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:57.848844 (+209388us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:57.968744 (+119900us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:57.991909 (+ 23165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23685,"tablet-delete.queue_time_us":650855}
I20260926 19:58:57.996559 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet d48ee26c0aaa4abc8e08a1a2dadb028d (table default.Table_1e071d1293234cecb72d6a118e8d5f10 [id=21d0c2efe78e438e8fad11d811c463ed]) successfully deleted
I20260926 19:58:57.996795 13569 tablet_replica.cc:333] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:57.999760 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:58.000317 12920 raft_consensus.cc:3060] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.008368 12920 raft_consensus.cc:2468] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:58.010313 13092 tablet_bootstrap.cc:492] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:58.010715 13092 ts_tablet_manager.cc:1403] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.089s user 0.006s sys 0.007s
I20260926 19:58:58.012686 12857 leader_election.cc:304] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:58.013360 13803 raft_consensus.cc:2804] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:58.013818 13803 raft_consensus.cc:697] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:58.014562 13803 consensus_queue.cc:237] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.015429 13092 raft_consensus.cc:359] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.015995 13092 raft_consensus.cc:385] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.016038 13569 raft_consensus.cc:2243] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:58.016235 13092 raft_consensus.cc:740] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.016781 13569 raft_consensus.cc:2272] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:58.021822 13092 consensus_queue.cc:260] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.027172 13092 ts_tablet_manager.cc:1434] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.016s user 0.007s sys 0.001s
I20260926 19:58:58.028158 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5930402db43740b4b291c6c238ecf52c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:58.028775 12920 raft_consensus.cc:2468] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:58.031319 13569 ts_tablet_manager.cc:1916] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:58.038767 13092 tablet_bootstrap.cc:492] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:58.043380 12390 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet e9c7b63db44742108dc132df5089e0d9
I20260926 19:58:58.044740 12390 catalog_manager.cc:6934] ReplaceTablet: tablet e9c7b63db44742108dc132df5089e0d9 deleted and replaced by tablet 2cc8bbccd0bf403db22e3791b2fb854d
I20260926 19:58:58.046344 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:58.046525 12895 tablet_service.cc:1558] Processing DeleteTablet for tablet e9c7b63db44742108dc132df5089e0d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2cc8bbccd0bf403db22e3791b2fb854d) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:58.050812 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet e9c7b63db44742108dc132df5089e0d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2cc8bbccd0bf403db22e3791b2fb854d) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:58.056308 13092 tablet_bootstrap.cc:654] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:58.048347 12392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 56) took 1044 ms (client timeout 1000 ms). Trace:
I20260926 19:58:58.075450 13840 raft_consensus.cc:493] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:58.076047 13840 raft_consensus.cc:515] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.079011 13840 leader_election.cc:290] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
W20260926 19:58:58.058579 12392 rpcz_store.cc:269] 0926 19:58:57.003990 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.021904 (+ 17914us) service_pool.cc:224] Handling call
0926 19:58:58.048285 (+1026381us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:57.530634 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:57.531021 (+ 387us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:57.531045 (+ 24us) write_op.cc:435] Acquired schema lock
0926 19:58:57.531057 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:57.531406 (+ 349us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:57.531444 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:57.531465 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:57.531616 (+ 151us) tablet.cc:701] Row locks acquired
0926 19:58:57.531627 (+ 11us) write_op.cc:260] PREPARE: finished
0926 19:58:57.531730 (+ 103us) write_op.cc:270] Start()
0926 19:58:57.531807 (+ 77us) write_op.cc:276] Timestamp: P: 1790452737531685 usec, L: 0
0926 19:58:57.531821 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:57.689610 (+157789us) log.cc:844] Serialized 536 byte log entry
0926 19:58:57.692343 (+ 2733us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36ac5a5e8 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11 00007fb36b0edd3d 00000000005bd4e4 00000000005bd319
0926 19:58:58.032719 (+340376us) op_driver.cc:464] REPLICATION: finished
0926 19:58:58.033705 (+ 986us) write_op.cc:301] APPLY: starting
0926 19:58:58.033763 (+ 58us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:58.033853 (+ 90us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:58.033871 (+ 18us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:58.034014 (+ 143us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:58.034138 (+ 124us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:58.034176 (+ 38us) write_op.cc:312] APPLY: finished
0926 19:58:58.034339 (+ 163us) log.cc:844] Serialized 23 byte log entry
0926 19:58:58.034495 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:58.034551 (+ 56us) write_op.cc:454] Released schema lock
0926 19:58:58.034579 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:58.034798 (+ 219us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:58.034821 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":828,"apply.run_cpu_time_us":1061,"apply.run_wall_time_us":1067,"num_ops":1,"prepare.queue_time_us":132,"prepare.run_cpu_time_us":4336,"prepare.run_wall_time_us":161844,"replication_time_us":500805,"spinlock_wait_cycles":361710976}]]}
I20260926 19:58:58.091430 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89870d2c5b0342c4bdf66474fa8d38fe" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:58.092034 12759 raft_consensus.cc:2468] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:58.094311 13840 raft_consensus.cc:493] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:58.094949 13840 raft_consensus.cc:515] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.097922 13840 leader_election.cc:290] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:58.104769 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad76fc959e24545bdfad19f988fce93" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:58.105476 12920 raft_consensus.cc:2468] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:58.114158 13092 tablet_bootstrap.cc:492] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:58.114684 13092 ts_tablet_manager.cc:1403] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.076s user 0.005s sys 0.006s
I20260926 19:58:58.103494 13012 tablet_bootstrap.cc:654] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.127364 13092 raft_consensus.cc:359] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.111898 12696 leader_election.cc:304] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:58.128736 13914 raft_consensus.cc:2804] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:58.129699 13914 raft_consensus.cc:697] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:58.130607 13914 consensus_queue.cc:237] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:58:58.139833 12395 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1b4aff22b9e4932beff6c2b90a9ab1e consensus state has no leader
I20260926 19:58:58.142695 12696 leader_election.cc:304] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:58.143026 12391 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet f70f8b9569884e8c8898f0f5e5771cd9
I20260926 19:58:58.143326 13092 raft_consensus.cc:385] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.143596 13092 raft_consensus.cc:740] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.145658 13915 raft_consensus.cc:2804] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:58.145985 13915 raft_consensus.cc:493] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:58.146318 13915 raft_consensus.cc:3060] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.144752 13012 tablet_bootstrap.cc:492] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:58.148113 13012 ts_tablet_manager.cc:1403] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.188s user 0.011s sys 0.001s
I20260926 19:58:58.148535 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_eb4069f7356a45b7bbbd58b430343d29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:58.150015 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb4069f7356a45b7bbbd58b430343d29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:58.150076 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_66bb1825f60742cc90be30615c83724a" } new_table_name: "default.Table_c7ed422235cf418792ba801b45f1f663" modify_external_catalogs: true
I20260926 19:58:58.154187 13012 raft_consensus.cc:359] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.154763 13012 raft_consensus.cc:385] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.154985 13012 raft_consensus.cc:740] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.155629 13012 consensus_queue.cc:260] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.157444 13012 ts_tablet_manager.cc:1434] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.009s user 0.002s sys 0.003s
I20260926 19:58:58.158344 13012 tablet_bootstrap.cc:492] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:58.174000 13403 ts_tablet_manager.cc:1929] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:58.174377 13403 log.cc:1199] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/c5b734f0d33342d3be8b1827bcd5dd14
I20260926 19:58:58.174432 13012 tablet_bootstrap.cc:654] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.174024 13915 raft_consensus.cc:515] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.176478 13915 leader_election.cc:290] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:58.180368 12696 leader_election.cc:304] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:58:58.181088 13840 raft_consensus.cc:2804] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:58.181406 13840 raft_consensus.cc:493] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:58.181808 13840 raft_consensus.cc:3060] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.181711 12857 leader_election.cc:304] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
W20260926 19:58:58.182214 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 74fe405570f8474fb7a1f841a783a90a (table default.Table_2246cb86fb8e49de90ee0350d6287a8c [id=2d2ae06595164e708751168edcdad523], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 74fe405570f8474fb7a1f841a783a90a consensus state has no leader
I20260926 19:58:58.182626 13882 raft_consensus.cc:2804] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:58.182965 13882 raft_consensus.cc:493] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:58.183265 13882 raft_consensus.cc:3060] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.188504 13840 raft_consensus.cc:515] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.190337 13882 raft_consensus.cc:515] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.192804 13882 leader_election.cc:290] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:58.193627 13092 consensus_queue.cc:260] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.194597 13840 leader_election.cc:290] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:58.195705 13092 ts_tablet_manager.cc:1434] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.081s user 0.005s sys 0.000s
I20260926 19:58:58.196574 13092 tablet_bootstrap.cc:492] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:58:58.203197 13012 tablet_bootstrap.cc:492] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:58.203677 13012 ts_tablet_manager.cc:1403] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.046s user 0.011s sys 0.000s
I20260926 19:58:58.215791 12591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89870d2c5b0342c4bdf66474fa8d38fe" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:58.218111 13012 raft_consensus.cc:359] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.218689 13012 raft_consensus.cc:385] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.218906 13012 raft_consensus.cc:740] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.219238 12556 tablet_service.cc:1558] Processing DeleteTablet for tablet e9c7b63db44742108dc132df5089e0d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 2cc8bbccd0bf403db22e3791b2fb854d) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:58.224169 12590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27c5b06d33c43ab9a483e1fc9b18234" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:58.224838 12590 raft_consensus.cc:2468] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
W20260926 19:58:58.228601 12695 leader_election.cc:343] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 5930402db43740b4b291c6c238ecf52c
W20260926 19:58:58.229593 12858 consensus_peers.cc:597] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: acdb4bd8cd58444cb1d6d4b5610de20d. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:58.217157 12591 raft_consensus.cc:2468] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:58.232211 12695 leader_election.cc:304] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:58.232524 13871 raft_consensus.cc:493] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:58.232926 13871 raft_consensus.cc:515] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.233408 13874 raft_consensus.cc:2804] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:58.233768 13874 raft_consensus.cc:493] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:58.234090 13874 raft_consensus.cc:3060] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.235526 13871 leader_election.cc:290] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:58.237722 13403 ts_tablet_manager.cc:1950] T c5b734f0d33342d3be8b1827bcd5dd14 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:58.239655 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 265) took 3056 ms. Trace:
I20260926 19:58:58.239931 13403 rpcz_store.cc:276] 0926 19:58:55.183625 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.183783 (+ 158us) service_pool.cc:224] Handling call
0926 19:58:57.663559 (+2479776us) ts_tablet_manager.cc:1151] Deleting tablet c5b734f0d33342d3be8b1827bcd5dd14
0926 19:58:57.663600 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:57.971265 (+307665us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:58.173864 (+202599us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:58.239597 (+ 65733us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2479229}
I20260926 19:58:58.241011 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet c5b734f0d33342d3be8b1827bcd5dd14 (table default.Table_2d039187041243c8adf2651cf58f0b4a [id=dba288eafec847baaf3732ce6d4292b5]) successfully deleted
I20260926 19:58:58.241120 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 266) took 2738 ms. Trace:
I20260926 19:58:58.241585 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89870d2c5b0342c4bdf66474fa8d38fe" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:58.242281 12594 raft_consensus.cc:3060] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.246701 13012 consensus_queue.cc:260] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.254860 12919 raft_consensus.cc:1275] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 2 FOLLOWER]: Refusing update from remote peer f6e9d6439fd54b28925a68a6145d3dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260926 19:58:58.256584 13012 ts_tablet_manager.cc:1434] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.053s user 0.005s sys 0.000s
I20260926 19:58:58.257436 13012 tablet_bootstrap.cc:492] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
W20260926 19:58:58.259553 12695 leader_election.cc:343] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 2ad76fc959e24545bdfad19f988fce93
I20260926 19:58:58.262557 13874 raft_consensus.cc:515] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.264786 13874 leader_election.cc:290] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:58.270223 13901 consensus_queue.cc:1048] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260926 19:58:58.276741 12858 leader_election.cc:343] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: b32f8b0d480547a58ba5c6fed6483a74
I20260926 19:58:58.280125 13013 tablet_bootstrap.cc:492] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:58.280745 13013 ts_tablet_manager.cc:1403] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.494s user 0.013s sys 0.003s
W20260926 19:58:58.280967 12857 consensus_peers.cc:597] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260926 19:58:58.281864 12695 leader_election.cc:343] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 2ad76fc959e24545bdfad19f988fce93
W20260926 19:58:58.282706 12695 leader_election.cc:343] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 5930402db43740b4b291c6c238ecf52c
I20260926 19:58:58.282651 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27c5b06d33c43ab9a483e1fc9b18234" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:58.283334 12919 raft_consensus.cc:2468] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
W20260926 19:58:58.283505 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet e9c7b63db44742108dc132df5089e0d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9c7b63db44742108dc132df5089e0d9 already in progress: creating tablet
I20260926 19:58:58.286707 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27c5b06d33c43ab9a483e1fc9b18234" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:58.286973 12594 raft_consensus.cc:2468] T 89870d2c5b0342c4bdf66474fa8d38fe P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:58.288491 12592 raft_consensus.cc:3060] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.290350 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_9725558d90dd4ba587d2c6e8def7f565"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:58.291899 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9725558d90dd4ba587d2c6e8def7f565 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:58.295598 12395 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a0c465b503b54e569fbf529da3ac316c has no consensus state
I20260926 19:58:58.298300 12760 raft_consensus.cc:1275] T 59d1955b12dd47899b8a226fe0e980ae P 3ae87963ed5e4d92a896bf4a73af4da9 [term 2 FOLLOWER]: Refusing update from remote peer f6e9d6439fd54b28925a68a6145d3dee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260926 19:58:58.298297 12858 leader_election.cc:304] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:58.299022 13882 raft_consensus.cc:2804] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:58.299435 13882 raft_consensus.cc:697] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:58.300186 13882 consensus_queue.cc:237] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.241391 13403 rpcz_store.cc:276] 0926 19:58:55.502337 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.502491 (+ 154us) service_pool.cc:224] Handling call
0926 19:58:58.240823 (+2738332us) ts_tablet_manager.cc:1151] Deleting tablet d734112eca1249559d0ff3d123e82042
0926 19:58:58.240859 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.241073 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2737714}
I20260926 19:58:58.303638 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 267) took 2719 ms. Trace:
I20260926 19:58:58.303905 13403 rpcz_store.cc:276] 0926 19:58:55.583876 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.584002 (+ 126us) service_pool.cc:224] Handling call
0926 19:58:58.303291 (+2719289us) ts_tablet_manager.cc:1151] Deleting tablet bfdfc9dd1b6046d18459a1255be03c4c
0926 19:58:58.303331 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.303589 (+ 258us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2669479}
I20260926 19:58:58.304944 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 268) took 2685 ms. Trace:
W20260926 19:58:58.306589 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet d734112eca1249559d0ff3d123e82042 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d734112eca1249559d0ff3d123e82042 already in progress: creating tablet
W20260926 19:58:58.307607 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet bfdfc9dd1b6046d18459a1255be03c4c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfdfc9dd1b6046d18459a1255be03c4c already in progress: creating tablet
I20260926 19:58:58.317212 12592 raft_consensus.cc:2468] T e27c5b06d33c43ab9a483e1fc9b18234 P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:58.322715 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5930402db43740b4b291c6c238ecf52c" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:58.323347 12919 raft_consensus.cc:3060] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.325078 13012 tablet_bootstrap.cc:654] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.331630 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad76fc959e24545bdfad19f988fce93" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:58.332175 12918 raft_consensus.cc:3060] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.305209 13403 rpcz_store.cc:276] 0926 19:58:55.619824 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.619957 (+ 133us) service_pool.cc:224] Handling call
0926 19:58:58.304669 (+2684712us) ts_tablet_manager.cc:1151] Deleting tablet deb7ea3bde5e4464a48b6e2843599f7d
0926 19:58:58.304707 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.304899 (+ 192us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2670568}
I20260926 19:58:58.334852 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 269) took 2544 ms. Trace:
I20260926 19:58:58.335121 13403 rpcz_store.cc:276] 0926 19:58:55.790442 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:55.790570 (+ 128us) service_pool.cc:224] Handling call
0926 19:58:58.334522 (+2543952us) ts_tablet_manager.cc:1151] Deleting tablet de85e3e17bf340f8895aff3f5421c85c
0926 19:58:58.334564 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.334802 (+ 238us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2543400}
I20260926 19:58:58.338544 12387 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_eca0d16a6176401c9021c2153e65b550"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:58.339569 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_640b34d9f5ac410a82cf9c2ba23cd525"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:58.340016 12387 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eca0d16a6176401c9021c2153e65b550 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:58:58.340893 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_640b34d9f5ac410a82cf9c2ba23cd525 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:58.345822 13012 tablet_bootstrap.cc:492] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:58.346194 13012 ts_tablet_manager.cc:1403] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.089s user 0.009s sys 0.000s
I20260926 19:58:58.347939 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_36d6c33f2b4f465f9f2b8823a09b3bfc" } modify_external_catalogs: true
I20260926 19:58:58.348438 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_36d6c33f2b4f465f9f2b8823a09b3bfc" } modify_external_catalogs: true
I20260926 19:58:58.349653 12918 raft_consensus.cc:2468] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:58.351508 12396 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_646a921cdd5c47d988e91966a807b79b" } new_table_name: "default.Table_98e4f6017318449bbfcf847f5d8b51ee" modify_external_catalogs: true
W20260926 19:58:58.352396 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet deb7ea3bde5e4464a48b6e2843599f7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet deb7ea3bde5e4464a48b6e2843599f7d already in progress: creating tablet
W20260926 19:58:58.353276 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet de85e3e17bf340f8895aff3f5421c85c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de85e3e17bf340f8895aff3f5421c85c already in progress: creating tablet
I20260926 19:58:58.353968 13012 raft_consensus.cc:359] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.354558 13012 raft_consensus.cc:385] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.354777 13012 raft_consensus.cc:740] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.355309 13012 consensus_queue.cc:260] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.357342 13012 ts_tablet_manager.cc:1434] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260926 19:58:58.362926 13012 tablet_bootstrap.cc:492] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:58.364768 12919 raft_consensus.cc:2468] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:58.374281 13012 tablet_bootstrap.cc:654] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:58:58.389735 13909 lock_manager.cc:585] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 4385111956292663372) tablet 00000000000000000000000000000000 cur holder 0x7b5800021000
I20260926 19:58:58.402666 13915 raft_consensus.cc:493] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:58.403241 13915 raft_consensus.cc:515] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.425014 13915 leader_election.cc:290] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:58.320811 13013 raft_consensus.cc:359] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.425712 13013 raft_consensus.cc:385] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.426144 13013 raft_consensus.cc:740] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.427104 13013 consensus_queue.cc:260] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.326033 13901 consensus_queue.cc:1048] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:58.406371 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f666e1a8d214172bf5b752e9ec374bd" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:58.431461 12595 raft_consensus.cc:2468] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:58.444051 13013 ts_tablet_manager.cc:1434] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.163s user 0.009s sys 0.002s
I20260926 19:58:58.452570 13013 tablet_bootstrap.cc:492] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:58.434219 12695 leader_election.cc:304] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:58.469731 12695 leader_election.cc:304] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:58.470508 13915 raft_consensus.cc:2804] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:58.470817 13915 raft_consensus.cc:493] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:58.471118 13915 raft_consensus.cc:3060] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.472008 13840 raft_consensus.cc:2804] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:58.472424 13840 raft_consensus.cc:697] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:58.473146 13840 consensus_queue.cc:237] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.473703 12391 master_service.cc:960] ReplaceTablet: received request to replace tablet 8f59be98ceef4effa15baf487c2d1140 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:58.412587 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e27c5b06d33c43ab9a483e1fc9b18234" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:58.474562 12918 raft_consensus.cc:3060] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.459784 13403 tablet_replica.cc:333] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:58.467818 12696 leader_election.cc:304] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:58.478889 13874 raft_consensus.cc:2804] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:58.479435 13874 raft_consensus.cc:697] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:58.480253 13874 consensus_queue.cc:237] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.495290 13403 raft_consensus.cc:2243] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:58.495823 13403 raft_consensus.cc:2272] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:58.499464 13403 ts_tablet_manager.cc:1916] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:58.520582 13013 tablet_bootstrap.cc:654] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.529093 13092 tablet_bootstrap.cc:654] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.564364 13013 tablet_bootstrap.cc:492] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:58.569648 13013 ts_tablet_manager.cc:1403] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.117s user 0.003s sys 0.008s
I20260926 19:58:58.571858 13013 raft_consensus.cc:359] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.575541 12918 raft_consensus.cc:2468] T e27c5b06d33c43ab9a483e1fc9b18234 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:58.581214 12392 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet c8179e29b84c4e148ce1e03be3555f53
I20260926 19:58:58.583112 13013 raft_consensus.cc:385] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.583374 13013 raft_consensus.cc:740] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.502271 13012 tablet_bootstrap.cc:492] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:58.585865 13012 ts_tablet_manager.cc:1403] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.223s user 0.025s sys 0.027s
I20260926 19:58:58.588116 13012 raft_consensus.cc:359] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.588698 13012 raft_consensus.cc:385] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.588925 13012 raft_consensus.cc:740] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.589483 13012 consensus_queue.cc:260] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.590590 13013 consensus_queue.cc:260] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.591392 13012 ts_tablet_manager.cc:1434] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260926 19:58:58.592191 13012 tablet_bootstrap.cc:492] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:58.592284 13013 ts_tablet_manager.cc:1434] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.022s user 0.004s sys 0.000s
I20260926 19:58:58.593082 13013 tablet_bootstrap.cc:492] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
W20260926 19:58:58.543092 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 55203.1us
I20260926 19:58:58.593909 12696 leader_election.cc:304] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:58.598242 13914 raft_consensus.cc:2804] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:58.598723 13914 raft_consensus.cc:697] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:58.599503 13914 consensus_queue.cc:237] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.603235 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89870d2c5b0342c4bdf66474fa8d38fe" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:58.603745 12759 raft_consensus.cc:3060] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.604732 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b32f8b0d480547a58ba5c6fed6483a74" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:58.605254 12757 raft_consensus.cc:2468] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:58:58.609014 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f666e1a8d214172bf5b752e9ec374bd" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:58.609593 12919 raft_consensus.cc:2468] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:58:58.610494 12759 raft_consensus.cc:2468] T 89870d2c5b0342c4bdf66474fa8d38fe P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:58.492579 13569 ts_tablet_manager.cc:1929] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:58.613735 13569 log.cc:1199] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/acdb4bd8cd58444cb1d6d4b5610de20d
I20260926 19:58:58.614604 13569 ts_tablet_manager.cc:1950] T acdb4bd8cd58444cb1d6d4b5610de20d P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:58.615377 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 300) took 1622 ms. Trace:
I20260926 19:58:58.615733 13569 rpcz_store.cc:276] 0926 19:58:56.992530 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.992657 (+ 127us) service_pool.cc:224] Handling call
0926 19:58:57.996649 (+1003992us) ts_tablet_manager.cc:1151] Deleting tablet acdb4bd8cd58444cb1d6d4b5610de20d
0926 19:58:57.996692 (+ 43us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.202583 (+205891us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:58.492469 (+289886us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:58.615329 (+122860us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1001873}
I20260926 19:58:58.616957 13901 raft_consensus.cc:493] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:58.617492 13901 raft_consensus.cc:515] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.619731 13901 leader_election.cc:290] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:58.620436 13013 tablet_bootstrap.cc:654] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.622890 12641 heartbeater.cc:344] Connected to a master server at 127.11.233.253:37913
I20260926 19:58:58.632124 12392 master_service.cc:960] ReplaceTablet: received request to replace tablet d734112eca1249559d0ff3d123e82042 from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:58.632326 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "700d05149a934d5e8bec9cc41873ad3e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:58.633738 12521 leader_election.cc:343] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:58.634861 13915 raft_consensus.cc:515] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.635324 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet acdb4bd8cd58444cb1d6d4b5610de20d (table default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]) successfully deleted
I20260926 19:58:58.635725 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 296) took 1663 ms. Trace:
I20260926 19:58:58.636044 13569 rpcz_store.cc:276] 0926 19:58:56.971851 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.991311 (+ 19460us) service_pool.cc:224] Handling call
0926 19:58:58.635477 (+1644166us) ts_tablet_manager.cc:1151] Deleting tablet 66c8916d88dd48998eab514bd3c538b9
0926 19:58:58.635525 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.635672 (+ 147us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1640413}
W20260926 19:58:58.637063 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 66c8916d88dd48998eab514bd3c538b9 because the tablet was not found. No further retry: Not found: Tablet not found: 66c8916d88dd48998eab514bd3c538b9
I20260926 19:58:58.637668 13915 leader_election.cc:290] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:58.621899 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "700d05149a934d5e8bec9cc41873ad3e" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:58.650926 12519 leader_election.cc:343] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:58.651300 12519 leader_election.cc:304] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:58.652323 13901 raft_consensus.cc:2749] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:58.622092 13012 tablet_bootstrap.cc:654] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.658330 12857 leader_election.cc:304] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:58.659392 13882 raft_consensus.cc:2804] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:58:58.659765 13882 raft_consensus.cc:493] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:58:58.660121 13882 raft_consensus.cc:3060] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.665879 13013 tablet_bootstrap.cc:492] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:58.666347 13013 ts_tablet_manager.cc:1403] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.073s user 0.010s sys 0.000s
I20260926 19:58:58.668623 13013 raft_consensus.cc:359] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.669622 13013 raft_consensus.cc:385] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.669407 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f666e1a8d214172bf5b752e9ec374bd" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:58:58.669981 12918 raft_consensus.cc:3060] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.673888 13569 tablet_replica.cc:333] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:58.674722 13569 raft_consensus.cc:2243] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:58.675710 13569 raft_consensus.cc:2272] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:58.680814 13882 raft_consensus.cc:515] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.673959 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f666e1a8d214172bf5b752e9ec374bd" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee"
I20260926 19:58:58.684194 12592 raft_consensus.cc:3060] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:58:58.684501 12858 leader_election.cc:343] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: b32f8b0d480547a58ba5c6fed6483a74
I20260926 19:58:58.691326 12592 raft_consensus.cc:2468] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:58.692178 13013 raft_consensus.cc:740] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.692902 13013 consensus_queue.cc:260] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.696048 13013 ts_tablet_manager.cc:1434] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.029s user 0.004s sys 0.000s
I20260926 19:58:58.696877 13569 ts_tablet_manager.cc:1916] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:58.707670 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b32f8b0d480547a58ba5c6fed6483a74" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:58:58.708166 12759 raft_consensus.cc:3060] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:58:58.713181 12695 leader_election.cc:304] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:58:58.715552 13914 raft_consensus.cc:2804] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:58.716030 13914 raft_consensus.cc:697] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:58:58.716806 13914 consensus_queue.cc:237] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.720389 13013 tablet_bootstrap.cc:492] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:58.720870 12759 raft_consensus.cc:2468] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:58:58.721940 12857 leader_election.cc:304] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:58.722756 13803 raft_consensus.cc:2804] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:58:58.723179 13803 raft_consensus.cc:697] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Becoming Leader. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Running, Role: LEADER
I20260926 19:58:58.723919 13803 consensus_queue.cc:237] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:58:58.728938 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_bfdeb281a21842bc9cf53aafa87f80cd" } new_table_name: "default.Table_ac3b4dbe4f0241d09ea35f62599b84a5" modify_external_catalogs: true
I20260926 19:58:58.731102 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_c8b681e9cf4b4c26a3ea0fb2f83c0cd6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:58.732942 13012 tablet_bootstrap.cc:492] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:58.733338 13012 ts_tablet_manager.cc:1403] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.141s user 0.006s sys 0.003s
I20260926 19:58:58.736932 13882 leader_election.cc:290] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:58.739527 13012 raft_consensus.cc:359] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.745635 13962 consensus_queue.cc:1048] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:58:58.758555 13012 raft_consensus.cc:385] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.738438 12918 raft_consensus.cc:2468] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:58:58.764945 13012 raft_consensus.cc:740] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
W20260926 19:58:58.766678 13957 log.cc:927] Time spent T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: Append to log took a long time: real 0.227s user 0.033s sys 0.000s
I20260926 19:58:58.767027 13403 ts_tablet_manager.cc:1929] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:58.767381 13403 log.cc:1199] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/caf4547f148f49b8943f4e1c7f2132a7
I20260926 19:58:58.768083 13403 ts_tablet_manager.cc:1950] T caf4547f148f49b8943f4e1c7f2132a7 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:58:58.768785 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 270) took 2533 ms. Trace:
W20260926 19:58:58.769656 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8b681e9cf4b4c26a3ea0fb2f83c0cd6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:58.776480 13012 consensus_queue.cc:260] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:58.782609 13013 tablet_bootstrap.cc:654] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.769073 13403 rpcz_store.cc:276] 0926 19:58:56.235504 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.235656 (+ 152us) service_pool.cc:224] Handling call
0926 19:58:58.335877 (+2100221us) ts_tablet_manager.cc:1151] Deleting tablet caf4547f148f49b8943f4e1c7f2132a7
0926 19:58:58.459667 (+123790us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.520077 (+ 60410us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:58.766907 (+246830us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:58.768737 (+ 1830us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19424,"tablet-delete.queue_time_us":2099173}
I20260926 19:58:58.786939 13012 ts_tablet_manager.cc:1434] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.053s user 0.005s sys 0.000s
I20260926 19:58:58.794027 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet caf4547f148f49b8943f4e1c7f2132a7 (table default.Table_724fda2a18204a24979779d6e0aafb81 [id=b89cc0604f6f4019802a9ce497b4b954]) successfully deleted
I20260926 19:58:58.795260 13012 tablet_bootstrap.cc:492] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:58.807117 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_c341da116f104491a6da32f46b955ecd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:58.808538 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c341da116f104491a6da32f46b955ecd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:58.818148 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ffee871354884e4a882a0945a6a111c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:58.819550 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ffee871354884e4a882a0945a6a111c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:58.825867 13013 tablet_bootstrap.cc:492] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:58.826262 13013 ts_tablet_manager.cc:1403] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.106s user 0.005s sys 0.006s
I20260926 19:58:58.828756 13013 raft_consensus.cc:359] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.834290 13012 tablet_bootstrap.cc:654] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.829370 13013 raft_consensus.cc:385] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.841704 13013 raft_consensus.cc:740] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.847525 13013 consensus_queue.cc:260] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.859524 13569 ts_tablet_manager.cc:1929] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:58.864209 13569 log.cc:1199] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/06c5ef9e1bca427fb889cbb17f77f999
I20260926 19:58:58.864279 13013 ts_tablet_manager.cc:1434] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260926 19:58:58.865255 13013 tablet_bootstrap.cc:492] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:58.863727 13012 tablet_bootstrap.cc:492] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:58.865775 13012 ts_tablet_manager.cc:1403] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 0.071s user 0.006s sys 0.004s
I20260926 19:58:58.868681 13012 raft_consensus.cc:359] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.887073 13012 raft_consensus.cc:385] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.890872 13012 raft_consensus.cc:740] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.892350 13012 consensus_queue.cc:260] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.869784 13569 ts_tablet_manager.cc:1950] T 06c5ef9e1bca427fb889cbb17f77f999 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:58:58.896440 13962 raft_consensus.cc:493] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:58.896884 13962 raft_consensus.cc:515] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.897997 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 301) took 1903 ms. Trace:
I20260926 19:58:58.898366 13569 rpcz_store.cc:276] 0926 19:58:56.994545 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.994661 (+ 116us) service_pool.cc:224] Handling call
0926 19:58:58.673742 (+1679081us) ts_tablet_manager.cc:1151] Deleting tablet 06c5ef9e1bca427fb889cbb17f77f999
0926 19:58:58.673784 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.804588 (+130804us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:58.859393 (+ 54805us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:58.897940 (+ 38547us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18734,"tablet-delete.queue_time_us":1678517}
I20260926 19:58:58.900074 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 302) took 1858 ms. Trace:
I20260926 19:58:58.900110 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 06c5ef9e1bca427fb889cbb17f77f999 (table default.Table_06d7f301c3cb4d0e8d374df103fab2f2 [id=ab940466309d4545b5a3f15ecc020742]) successfully deleted
I20260926 19:58:58.900421 13569 rpcz_store.cc:276] 0926 19:58:57.041606 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.050219 (+ 8613us) service_pool.cc:224] Handling call
0926 19:58:58.899756 (+1849537us) ts_tablet_manager.cc:1151] Deleting tablet a26e6b837f67445c9383599b7e52de7b
0926 19:58:58.899798 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.900026 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1849000}
W20260926 19:58:58.901552 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet a26e6b837f67445c9383599b7e52de7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a26e6b837f67445c9383599b7e52de7b already in progress: creating tablet
I20260926 19:58:58.901741 13013 tablet_bootstrap.cc:654] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:58.901893 13962 leader_election.cc:290] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:58.903540 13012 ts_tablet_manager.cc:1434] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.037s user 0.004s sys 0.001s
I20260926 19:58:58.904402 13012 tablet_bootstrap.cc:492] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:58:58.928800 12393 catalog_manager.cc:5838] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee reported cstate change: term changed from 0 to 2, leader changed from <none> to f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193). New cstate: current_term: 2 leader_uuid: "f6e9d6439fd54b28925a68a6145d3dee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:58.930047 12393 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:58.933641 12388 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1cd57fde79174a2e8092337be2e108c1" } modify_external_catalogs: true
I20260926 19:58:58.934126 12388 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1cd57fde79174a2e8092337be2e108c1" } modify_external_catalogs: true
I20260926 19:58:58.939365 12731 tablet_service.cc:1511] Processing CreateTablet for tablet cde094c41f8d4a4e8b2f34cea2825326 (DEFAULT_TABLE table=default.Table_ecc2264951a44e51afb66b0053bc0474 [id=a155af54316b4028a30d69df50df5b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:58.939988 12556 tablet_service.cc:1511] Processing CreateTablet for tablet cde094c41f8d4a4e8b2f34cea2825326 (DEFAULT_TABLE table=default.Table_ecc2264951a44e51afb66b0053bc0474 [id=a155af54316b4028a30d69df50df5b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:58.941079 12731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cde094c41f8d4a4e8b2f34cea2825326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:58.942045 12556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cde094c41f8d4a4e8b2f34cea2825326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:58.942014 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 304) took 1420 ms. Trace:
I20260926 19:58:58.942314 13569 rpcz_store.cc:276] 0926 19:58:57.521352 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.524350 (+ 2998us) service_pool.cc:224] Handling call
0926 19:58:58.941671 (+1417321us) ts_tablet_manager.cc:1151] Deleting tablet 9411d3bca81641a4b3dfaffb2dc2bb51
0926 19:58:58.941717 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.941962 (+ 245us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1405510}
I20260926 19:58:58.942524 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_5ccfc6e2d4a84071bc4376d8c425a135"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:58:58.943413 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 305) took 1276 ms. Trace:
I20260926 19:58:58.943678 13569 rpcz_store.cc:276] 0926 19:58:57.667107 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.669811 (+ 2704us) service_pool.cc:224] Handling call
0926 19:58:58.943118 (+1273307us) ts_tablet_manager.cc:1151] Deleting tablet cfea8dbdd1ba49a5bd9d17652eb271c3
0926 19:58:58.943155 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.943365 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1272749}
W20260926 19:58:58.943986 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ccfc6e2d4a84071bc4376d8c425a135 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:58.944684 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 306) took 1258 ms. Trace:
I20260926 19:58:58.944948 13569 rpcz_store.cc:276] 0926 19:58:57.686610 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.686747 (+ 137us) service_pool.cc:224] Handling call
0926 19:58:58.944400 (+1257653us) ts_tablet_manager.cc:1151] Deleting tablet 54553d20d48148ae88fa4cc2ce021779
0926 19:58:58.944436 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.944630 (+ 194us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1257129}
I20260926 19:58:58.946065 13569 tablet_replica.cc:333] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
W20260926 19:58:58.947041 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9411d3bca81641a4b3dfaffb2dc2bb51 already in progress: creating tablet
W20260926 19:58:58.947695 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfea8dbdd1ba49a5bd9d17652eb271c3 already in progress: creating tablet
W20260926 19:58:58.948232 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 54553d20d48148ae88fa4cc2ce021779 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54553d20d48148ae88fa4cc2ce021779 already in progress: creating tablet
I20260926 19:58:58.948989 13013 tablet_bootstrap.cc:492] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:58.949435 13013 ts_tablet_manager.cc:1403] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.084s user 0.009s sys 0.002s
I20260926 19:58:58.954533 13013 raft_consensus.cc:359] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.955482 13013 raft_consensus.cc:385] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:58.955703 12392 master_service.cc:960] ReplaceTablet: received request to replace tablet 01d8740b03594023a8420c7a7121b80c from {username='slave'} at 127.0.0.1:45734
I20260926 19:58:58.955796 13013 raft_consensus.cc:740] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:58.956699 13013 consensus_queue.cc:260] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:58.958253 12555 tablet_service.cc:1511] Processing CreateTablet for tablet b29ebe0d128d4d1d925e8a01a3a0b70c (DEFAULT_TABLE table=default.Table_564a067586d540a6919f287941cf0392 [id=aa487600e10140a792e5a8fc0763de98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:58.959860 12555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b29ebe0d128d4d1d925e8a01a3a0b70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:58.973152 13569 raft_consensus.cc:2243] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:58:58.973764 13569 pending_rounds.cc:70] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Trying to abort 1 pending ops.
I20260926 19:58:58.973995 13569 pending_rounds.cc:77] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694414667137024 op_type: NO_OP noop_request { }
I20260926 19:58:58.974366 13569 raft_consensus.cc:2889] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:58.974716 13569 raft_consensus.cc:2272] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:58.982028 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 273) took 2577 ms. Trace:
I20260926 19:58:58.982367 13403 rpcz_store.cc:276] 0926 19:58:56.404517 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.408183 (+ 3666us) service_pool.cc:224] Handling call
0926 19:58:58.796168 (+2387985us) ts_tablet_manager.cc:1151] Deleting tablet 4d153e60ae5d486cb91c2cc623b99985
0926 19:58:58.796208 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:58.981946 (+185738us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2387436}
I20260926 19:58:58.982658 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 83d1df73b73c498081bfc4b29ae95448 (DEFAULT_TABLE table=default.Table_2079ba9cd2c7475a8046395706e15d9f [id=5bec506a57fb4509ab1282bb4083059e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:58.984297 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d1df73b73c498081bfc4b29ae95448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.001114 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 83d1df73b73c498081bfc4b29ae95448 (DEFAULT_TABLE table=default.Table_2079ba9cd2c7475a8046395706e15d9f [id=5bec506a57fb4509ab1282bb4083059e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.002183 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d1df73b73c498081bfc4b29ae95448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.005744 13926 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:58:59.012940 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 83d1df73b73c498081bfc4b29ae95448 (DEFAULT_TABLE table=default.Table_2079ba9cd2c7475a8046395706e15d9f [id=5bec506a57fb4509ab1282bb4083059e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.014015 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83d1df73b73c498081bfc4b29ae95448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.014645 13915 raft_consensus.cc:493] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:59.015084 13915 raft_consensus.cc:515] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:59.015244 12735 tablet_service.cc:1511] Processing CreateTablet for tablet b29ebe0d128d4d1d925e8a01a3a0b70c (DEFAULT_TABLE table=default.Table_564a067586d540a6919f287941cf0392 [id=aa487600e10140a792e5a8fc0763de98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.016273 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b29ebe0d128d4d1d925e8a01a3a0b70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.006335 13926 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:58:59.025517 13013 ts_tablet_manager.cc:1434] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.074s user 0.007s sys 0.000s
I20260926 19:58:59.027130 13013 tablet_bootstrap.cc:492] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:59.028122 13886 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a69f5870f8d461aa5e61f6999bb05c6. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:58:59.028971 13886 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:58:59.029477 13901 raft_consensus.cc:493] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:59.029968 13901 raft_consensus.cc:515] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.032182 13901 leader_election.cc:290] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:59.036554 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 5d019d9e2bab4fd3a92b921d972a4cf3 (DEFAULT_TABLE table=default.Table_d20824670f7d4516b7387bc0d28da6dd [id=c44794014cac46bd97baf193a05ce57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.037643 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d019d9e2bab4fd3a92b921d972a4cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.037833 12897 tablet_service.cc:1511] Processing CreateTablet for tablet cde094c41f8d4a4e8b2f34cea2825326 (DEFAULT_TABLE table=default.Table_ecc2264951a44e51afb66b0053bc0474 [id=a155af54316b4028a30d69df50df5b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.037978 13986 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 19:58:59.038290 12568 tablet_service.cc:1511] Processing CreateTablet for tablet a1e776237ca24a32a8e73d2f0628df3e (DEFAULT_TABLE table=default.Table_4144e2041c67465ea697a20aa6b1800b [id=b4bf3d9f4ea24cd6923d11d90cbe1b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.038753 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cde094c41f8d4a4e8b2f34cea2825326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.037266 12561 tablet_service.cc:1511] Processing CreateTablet for tablet 5d019d9e2bab4fd3a92b921d972a4cf3 (DEFAULT_TABLE table=default.Table_d20824670f7d4516b7387bc0d28da6dd [id=c44794014cac46bd97baf193a05ce57c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:59.043841 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 4d153e60ae5d486cb91c2cc623b99985 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d153e60ae5d486cb91c2cc623b99985 already in progress: creating tablet
I20260926 19:58:59.048640 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 276) took 2574 ms. Trace:
I20260926 19:58:59.048991 13403 rpcz_store.cc:276] 0926 19:58:56.474529 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.474670 (+ 141us) service_pool.cc:224] Handling call
0926 19:58:59.040297 (+2565627us) ts_tablet_manager.cc:1151] Deleting tablet 53af06be466d47569d5aa15727dadd0a
0926 19:58:59.040348 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:59.048581 (+ 8233us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2565060}
I20260926 19:58:59.052974 13901 raft_consensus.cc:493] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:59.053447 13901 raft_consensus.cc:515] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:59.056960 13901 leader_election.cc:290] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:59.071779 13926 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:58:59.078560 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e776237ca24a32a8e73d2f0628df3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.082890 13013 tablet_bootstrap.cc:654] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:59.086031 12561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d019d9e2bab4fd3a92b921d972a4cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.090384 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d8740b03594023a8420c7a7121b80c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:59.091711 12521 leader_election.cc:343] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:59.099166 13013 tablet_bootstrap.cc:492] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:59.099587 13013 ts_tablet_manager.cc:1403] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.073s user 0.010s sys 0.008s
I20260926 19:58:59.101784 13013 raft_consensus.cc:359] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.102474 13013 raft_consensus.cc:385] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:59.104746 13569 ts_tablet_manager.cc:1916] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:59.108393 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7dc92330d400f93f427f3f79ad826" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
W20260926 19:58:59.113658 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 71095us
I20260926 19:58:59.114275 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986a28107bd54199bf2cda96978150c3" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:59.102725 13013 raft_consensus.cc:740] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:59.116742 13013 consensus_queue.cc:260] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.125296 13013 ts_tablet_manager.cc:1434] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20260926 19:58:59.126165 13013 tablet_bootstrap.cc:492] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:59.127724 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 34214ac9d71b4fa583a5bd3128fd4981 (DEFAULT_TABLE table=default.Table_f42b2a59275343c796ebf6f3e3e96154 [id=4b3ebe94018741cc81697137b80490e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.129467 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34214ac9d71b4fa583a5bd3128fd4981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.130285 12891 tablet_service.cc:1511] Processing CreateTablet for tablet b29ebe0d128d4d1d925e8a01a3a0b70c (DEFAULT_TABLE table=default.Table_564a067586d540a6919f287941cf0392 [id=aa487600e10140a792e5a8fc0763de98]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:59.131440 12521 leader_election.cc:343] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:59.132198 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 277) took 2570 ms. Trace:
I20260926 19:58:59.132496 13403 rpcz_store.cc:276] 0926 19:58:56.561725 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.574112 (+ 12387us) service_pool.cc:224] Handling call
0926 19:58:59.054503 (+2480391us) ts_tablet_manager.cc:1151] Deleting tablet 90b7a9622b0f412b9a88aa84419ad0d7
0926 19:58:59.131918 (+ 77415us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:59.132146 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2348425}
I20260926 19:58:59.133625 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 283) took 2431 ms. Trace:
I20260926 19:58:59.133919 13403 rpcz_store.cc:276] 0926 19:58:56.701969 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.709596 (+ 7627us) service_pool.cc:224] Handling call
0926 19:58:59.133276 (+2423680us) ts_tablet_manager.cc:1151] Deleting tablet 71065b80482442699f4b207cb8f9d40c
0926 19:58:59.133312 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:59.133577 (+ 265us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2423199}
I20260926 19:58:59.134825 13403 tablet_replica.cc:333] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:58:59.135524 13403 raft_consensus.cc:2243] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:59.135959 13403 raft_consensus.cc:2272] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:59.139266 13403 ts_tablet_manager.cc:1916] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:59.144712 12891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b29ebe0d128d4d1d925e8a01a3a0b70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.144622 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 34214ac9d71b4fa583a5bd3128fd4981 (DEFAULT_TABLE table=default.Table_f42b2a59275343c796ebf6f3e3e96154 [id=4b3ebe94018741cc81697137b80490e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.149307 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 3409a06964ba45069360f60a8c5d910d (DEFAULT_TABLE table=default.Table_a1fcffb35ba74139a265e901aba3ae6a [id=3a28b516c6c045e6a7f6f8a9cc8f1dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.150345 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3409a06964ba45069360f60a8c5d910d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.161546 12729 tablet_service.cc:1511] Processing CreateTablet for tablet a1e776237ca24a32a8e73d2f0628df3e (DEFAULT_TABLE table=default.Table_4144e2041c67465ea697a20aa6b1800b [id=b4bf3d9f4ea24cd6923d11d90cbe1b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.162922 13915 leader_election.cc:290] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:59.166886 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 3409a06964ba45069360f60a8c5d910d (DEFAULT_TABLE table=default.Table_a1fcffb35ba74139a265e901aba3ae6a [id=3a28b516c6c045e6a7f6f8a9cc8f1dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.167510 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01d8740b03594023a8420c7a7121b80c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
W20260926 19:58:59.168745 12519 leader_election.cc:343] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:59.169152 12519 leader_election.cc:304] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:59.170672 13901 raft_consensus.cc:2749] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:59.170349 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 5d019d9e2bab4fd3a92b921d972a4cf3 (DEFAULT_TABLE table=default.Table_d20824670f7d4516b7387bc0d28da6dd [id=c44794014cac46bd97baf193a05ce57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.170409 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986a28107bd54199bf2cda96978150c3" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:59.175172 13985 consensus_queue.cc:1048] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:58:59.178428 12519 leader_election.cc:343] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:59.178810 12519 leader_election.cc:304] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:59.179788 13901 raft_consensus.cc:2749] T 986a28107bd54199bf2cda96978150c3 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:58:59.181595 13092 tablet_bootstrap.cc:492] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:58:59.182031 13092 ts_tablet_manager.cc:1403] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.986s user 0.010s sys 0.015s
I20260926 19:58:59.168548 13960 consensus_queue.cc:1048] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260926 19:58:59.194169 12565 tablet_service.cc:1511] Processing CreateTablet for tablet 342cc84ea5aa45259bf46068930eb921 (DEFAULT_TABLE table=default.Table_7c45f0a5bb2f469c88cc36ea50f3affe [id=687165a0167d461fb0e65b1ae6480c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.195192 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342cc84ea5aa45259bf46068930eb921. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:59.206142 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 53af06be466d47569d5aa15727dadd0a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 53af06be466d47569d5aa15727dadd0a already in progress: creating tablet
W20260926 19:58:59.208333 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 90b7a9622b0f412b9a88aa84419ad0d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90b7a9622b0f412b9a88aa84419ad0d7 already in progress: creating tablet
W20260926 19:58:59.210882 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 71065b80482442699f4b207cb8f9d40c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71065b80482442699f4b207cb8f9d40c already in progress: creating tablet
I20260926 19:58:59.212340 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 342cc84ea5aa45259bf46068930eb921 (DEFAULT_TABLE table=default.Table_7c45f0a5bb2f469c88cc36ea50f3affe [id=687165a0167d461fb0e65b1ae6480c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.213406 12897 tablet_service.cc:1511] Processing CreateTablet for tablet a1e776237ca24a32a8e73d2f0628df3e (DEFAULT_TABLE table=default.Table_4144e2041c67465ea697a20aa6b1800b [id=b4bf3d9f4ea24cd6923d11d90cbe1b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.214682 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e776237ca24a32a8e73d2f0628df3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.220178 13012 tablet_bootstrap.cc:654] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:59.221733 13092 raft_consensus.cc:359] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.222460 13092 raft_consensus.cc:385] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:59.222744 13092 raft_consensus.cc:740] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:58:59.223340 13092 consensus_queue.cc:260] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.232633 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad59d3cab654186ac2cbd3860980d01 (DEFAULT_TABLE table=default.Table_86ffbd3fac614b7db1a61cba6ab82328 [id=bce88e98599e43b78774bd03788b20e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.233989 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad59d3cab654186ac2cbd3860980d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.240052 12730 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad59d3cab654186ac2cbd3860980d01 (DEFAULT_TABLE table=default.Table_86ffbd3fac614b7db1a61cba6ab82328 [id=bce88e98599e43b78774bd03788b20e5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:58:59.243191 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 74fe405570f8474fb7a1f841a783a90a (table default.Table_2246cb86fb8e49de90ee0350d6287a8c [id=2d2ae06595164e708751168edcdad523], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 74fe405570f8474fb7a1f841a783a90a consensus state has no leader
I20260926 19:58:59.244465 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34214ac9d71b4fa583a5bd3128fd4981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.249668 12560 tablet_service.cc:1511] Processing CreateTablet for tablet 43a92aa343944eab9c937aeeac953649 (DEFAULT_TABLE table=default.Table_c360f50af2554b0984e0b30eb5bbd833 [id=0fafc84100524ebe8f48618a6c3b9234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.250046 12900 tablet_service.cc:1511] Processing CreateTablet for tablet 3409a06964ba45069360f60a8c5d910d (DEFAULT_TABLE table=default.Table_a1fcffb35ba74139a265e901aba3ae6a [id=3a28b516c6c045e6a7f6f8a9cc8f1dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.251134 12898 tablet_service.cc:1511] Processing CreateTablet for tablet 342cc84ea5aa45259bf46068930eb921 (DEFAULT_TABLE table=default.Table_7c45f0a5bb2f469c88cc36ea50f3affe [id=687165a0167d461fb0e65b1ae6480c00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.251899 12560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a92aa343944eab9c937aeeac953649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.252707 12893 tablet_service.cc:1511] Processing CreateTablet for tablet 34214ac9d71b4fa583a5bd3128fd4981 (DEFAULT_TABLE table=default.Table_f42b2a59275343c796ebf6f3e3e96154 [id=4b3ebe94018741cc81697137b80490e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.258846 13787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41610 (request call id 0) took 3102 ms. Trace:
I20260926 19:58:59.267863 13768 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" instance_seqno: 1790452709547740) as {username='slave'} at 127.11.233.193:60539; Asking this server to re-register.
I20260926 19:58:59.281728 13092 ts_tablet_manager.cc:1434] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.099s user 0.010s sys 0.001s
W20260926 19:58:59.288198 12393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 61) took 1830 ms (client timeout 1000 ms). Trace:
W20260926 19:58:59.290726 12696 leader_election.cc:343] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:59.291458 12389 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49]). Expected version 1 got 0
W20260926 19:58:59.288573 12393 rpcz_store.cc:269] 0926 19:58:57.457882 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.499028 (+ 41146us) service_pool.cc:224] Handling call
0926 19:58:57.682023 (+182995us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:59.288124 (+1606101us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:59.238950 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:59.239286 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:59.239308 (+ 22us) write_op.cc:435] Acquired schema lock
0926 19:58:59.239320 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:59.239544 (+ 224us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:59.239583 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:59.239603 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:59.239735 (+ 132us) tablet.cc:701] Row locks acquired
0926 19:58:59.239747 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:59.239844 (+ 97us) write_op.cc:270] Start()
0926 19:58:59.239935 (+ 91us) write_op.cc:276] Timestamp: P: 1790452739239761 usec, L: 0
0926 19:58:59.239950 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:59.240394 (+ 444us) log.cc:844] Serialized 536 byte log entry
0926 19:58:59.283735 (+ 43341us) op_driver.cc:464] REPLICATION: finished
0926 19:58:59.284185 (+ 450us) write_op.cc:301] APPLY: starting
0926 19:58:59.284248 (+ 63us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:59.284345 (+ 97us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:59.284464 (+ 119us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:59.284668 (+ 204us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:59.284868 (+ 200us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:59.284936 (+ 68us) write_op.cc:312] APPLY: finished
0926 19:58:59.285163 (+ 227us) log.cc:844] Serialized 23 byte log entry
0926 19:58:59.285304 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:59.285378 (+ 74us) write_op.cc:454] Released schema lock
0926 19:58:59.285422 (+ 44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":416209280,"child_traces":[["op",{"apply.queue_time_us":280,"apply.run_cpu_time_us":1477,"apply.run_wall_time_us":1477,"num_ops":1,"prepare.queue_time_us":307360,"raft.queue_time_us":259,"raft.run_cpu_time_us":1042,"raft.run_wall_time_us":1043,"replication_time_us":43685}]]}
I20260926 19:58:59.298825 12390 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:58:59.298977 12393 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_abeab641cddf49ca82c394e6e2072af4" } new_table_name: "default.Table_64c355e960994a9197d1f606cf6017a3" modify_external_catalogs: true
I20260926 19:58:59.299635 13569 ts_tablet_manager.cc:1929] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:59.300027 13569 log.cc:1199] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/89c8cbb097a644218e0ca033a6fdb4a2
I20260926 19:58:59.299973 12390 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:58:59.300846 13569 ts_tablet_manager.cc:1950] T 89c8cbb097a644218e0ca033a6fdb4a2 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
W20260926 19:58:59.301580 12389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 69) took 1719 ms (client timeout 1000 ms). Trace:
W20260926 19:58:59.301925 12389 rpcz_store.cc:269] 0926 19:58:57.582326 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.658574 (+ 76248us) service_pool.cc:224] Handling call
0926 19:58:57.680826 (+ 22252us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:59.301486 (+1620660us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":44823808}
I20260926 19:58:59.303175 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_33ac50d900914f2982f1880450cd2b5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:59.304697 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33ac50d900914f2982f1880450cd2b5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:59.306380 12390 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49]). Expected version 1 got 0
I20260926 19:58:59.311764 12391 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:59.312997 12391 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 986a28107bd54199bf2cda96978150c3 (table default.Table_646a921cdd5c47d988e91966a807b79b [id=948f2bdffbc84b0ebf0f88e273f0a1fe]). Expected version 1 got 0
W20260926 19:58:59.313995 12396 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 986a28107bd54199bf2cda96978150c3 consensus state has no leader
I20260926 19:58:59.315059 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_fc73e336739940449da359776b1f4c50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:59.318792 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fc73e336739940449da359776b1f4c50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:59.326668 12387 catalog_manager.cc:5838] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195). New cstate: current_term: 1 leader_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: UNKNOWN } } }
I20260926 19:58:59.328768 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49]). Expected version 1 got 0
I20260926 19:58:59.329402 12387 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:58:59.330235 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 6f666e1a8d214172bf5b752e9ec374bd
I20260926 19:58:59.331676 12394 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:45734 (request call id 383) took 1452 ms. Trace:
I20260926 19:58:59.335062 13092 tablet_bootstrap.cc:492] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
W20260926 19:58:59.245138 12858 consensus_peers.cc:597] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 22f268b4a1f7423f917731c6b62aab3e. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:58:59.306457 12900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3409a06964ba45069360f60a8c5d910d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.336434 12893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34214ac9d71b4fa583a5bd3128fd4981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.331979 12394 rpcz_store.cc:276] 0926 19:58:57.879382 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.879486 (+ 104us) service_pool.cc:224] Handling call
0926 19:58:57.880709 (+ 1223us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:57.881227 (+ 518us) catalog_manager.cc:2854] Modifying in-memory table state
0926 19:58:57.881365 (+ 138us) catalog_manager.cc:2863] Locking tablets
0926 19:58:58.926684 (+1045319us) catalog_manager.cc:2877] Removing table and tablets from system table
0926 19:58:59.067399 (+140715us) catalog_manager.cc:2895] Removing table from by-name map
0926 19:58:59.067599 (+ 200us) catalog_manager.cc:2911] Committing in-memory state
0926 19:58:59.330135 (+262536us) catalog_manager.cc:2915] Aborting table tasks
0926 19:58:59.331620 (+ 1485us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:58.929074 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:58.929429 (+ 355us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:58.929452 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:58:58.929464 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:58.929869 (+ 405us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:58.929904 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:58.929935 (+ 31us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:58:58.930200 (+ 265us) tablet.cc:701] Row locks acquired
0926 19:58:58.930213 (+ 13us) write_op.cc:260] PREPARE: finished
0926 19:58:58.930314 (+ 101us) write_op.cc:270] Start()
0926 19:58:58.930385 (+ 71us) write_op.cc:276] Timestamp: P: 1790452738930247 usec, L: 0
0926 19:58:58.930399 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:58:59.055015 (+124616us) log.cc:844] Serialized 765 byte log entry
0926 19:58:59.057784 (+ 2769us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36ac5a5e8 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11 00007fb36b0edd3d 00000000005bd4e4 00000000005bd319
0926 19:58:59.065733 (+ 7949us) op_driver.cc:464] REPLICATION: finished
0926 19:58:59.066118 (+ 385us) write_op.cc:301] APPLY: starting
0926 19:58:59.066178 (+ 60us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:59.066287 (+ 109us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:59.066307 (+ 20us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:59.066556 (+ 249us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:59.066705 (+ 149us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:58:59.066783 (+ 78us) write_op.cc:312] APPLY: finished
0926 19:58:59.066975 (+ 192us) log.cc:844] Serialized 29 byte log entry
0926 19:58:59.067122 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:59.067186 (+ 64us) write_op.cc:454] Released schema lock
0926 19:58:59.067217 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:59.238638 (+171421us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:59.238666 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":242,"apply.run_cpu_time_us":1294,"apply.run_wall_time_us":1294,"num_ops":2,"prepare.queue_time_us":769,"prepare.run_cpu_time_us":3640,"prepare.run_wall_time_us":309540,"raft.queue_time_us":172,"raft.run_cpu_time_us":681,"raft.run_wall_time_us":681,"replication_time_us":135236,"spinlock_wait_cycles":272497408}]]}
I20260926 19:58:59.337918 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342cc84ea5aa45259bf46068930eb921. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:59.338938 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 93848.1us
I20260926 19:58:59.347764 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 839b20750eba4d8185bc7057e37da5fe (DEFAULT_TABLE table=default.Table_46c845de9a6f4e368a893c401a7f43d2 [id=68d798ba84c54907b518ffbb6bb1064e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.363109 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 9599f427a64f40d7b185b23a849b46f3) from {username='slave'} at 127.0.0.1:37792
W20260926 19:58:59.363741 12390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 57) took 1338 ms (client timeout 1000 ms). Trace:
W20260926 19:58:59.365960 12390 rpcz_store.cc:269] 0926 19:58:58.025270 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:58.045392 (+ 20122us) service_pool.cc:224] Handling call
0926 19:58:58.135732 (+ 90340us) spinlock_profiling.cc:243] Waited 59.3 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:58:59.363677 (+1227945us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":15,"spinlock_wait_cycles":136416640}
I20260926 19:58:59.376713 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "700d05149a934d5e8bec9cc41873ad3e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:59.380460 13013 tablet_bootstrap.cc:654] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:59.411823 14002 tablet_replica.cc:333] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:59.413115 14002 raft_consensus.cc:2243] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
W20260926 19:58:59.414667 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 123074us
W20260926 19:58:59.417029 12696 leader_election.cc:343] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:59.420933 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 8d172eb62e144cc991370198c608ab58 (DEFAULT_TABLE table=default.Table_da82d46a08d745ddaebd2a7c79b1d4be [id=843400c90a554f839014213cd53c1379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.423831 14002 raft_consensus.cc:2272] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:59.427377 12728 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c5ef9e1bca427fb889cbb17f77f999 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:44 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:59.428583 12728 tablet_service.cc:1511] Processing CreateTablet for tablet 83a78d487f0147fb95e8b946539ab3a6 (DEFAULT_TABLE table=default.Table_94fb4afa42c74fa1b7ec938574559b46 [id=ba86aa08529b4135a224b8d53ffc7ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.428939 12904 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad59d3cab654186ac2cbd3860980d01 (DEFAULT_TABLE table=default.Table_86ffbd3fac614b7db1a61cba6ab82328 [id=bce88e98599e43b78774bd03788b20e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.429471 12727 tablet_service.cc:1558] Processing DeleteTablet for tablet 22f268b4a1f7423f917731c6b62aab3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:49 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:59.430583 12727 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb9c1e977f942d9b6e3a00c3f4bc694 (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.431149 12904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad59d3cab654186ac2cbd3860980d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.431398 12726 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ad76fc959e24545bdfad19f988fce93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:59.432468 12726 tablet_service.cc:1511] Processing CreateTablet for tablet a1d5deb0364542a7945e5675ca31557c (DEFAULT_TABLE table=default.Table_85695893a5b24687959b83a8360d1443 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.433431 12725 tablet_service.cc:1558] Processing DeleteTablet for tablet 5930402db43740b4b291c6c238ecf52c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:59.435092 12725 tablet_service.cc:1511] Processing CreateTablet for tablet 9ed22219df2b42ad86a6d7917ee0d990 (DEFAULT_TABLE table=default.Table_174debbe544e47ccbcf8faef177fdd0b [id=1a610b5d22a14bd79b983901002df704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.435571 14002 ts_tablet_manager.cc:1916] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:59.436681 12724 tablet_service.cc:1511] Processing CreateTablet for tablet 636169e8ee334ac4898973c33735b763 (DEFAULT_TABLE table=default.Table_94d1edd7294d4dc38229784cab1912ea [id=02e8cd9af9cc4dc98eced0e23adc461f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.436734 12556 tablet_service.cc:1511] Processing CreateTablet for tablet 8d172eb62e144cc991370198c608ab58 (DEFAULT_TABLE table=default.Table_da82d46a08d745ddaebd2a7c79b1d4be [id=843400c90a554f839014213cd53c1379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.437793 12556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d172eb62e144cc991370198c608ab58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.438639 12723 tablet_service.cc:1511] Processing CreateTablet for tablet a0c465b503b54e569fbf529da3ac316c (DEFAULT_TABLE table=default.Table_c7ed422235cf418792ba801b45f1f663 [id=b1b956e2eb0740bab149e499b6707263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.440234 12722 tablet_service.cc:1511] Processing CreateTablet for tablet 036b9a2a0d65468ab8ff19ba06ec72a0 (DEFAULT_TABLE table=default.Table_6787fdb6bfad44bc8eb2f6c7c0848f48 [id=dc9290fc992b47229a40c49469284183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.445955 13092 tablet_bootstrap.cc:654] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:59.446890 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f666e1a8d214172bf5b752e9ec374bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:59.450124 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 43a92aa343944eab9c937aeeac953649 (DEFAULT_TABLE table=default.Table_c360f50af2554b0984e0b30eb5bbd833 [id=0fafc84100524ebe8f48618a6c3b9234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.451104 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a92aa343944eab9c937aeeac953649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.443689 12565 tablet_service.cc:1511] Processing CreateTablet for tablet a0c465b503b54e569fbf529da3ac316c (DEFAULT_TABLE table=default.Table_c7ed422235cf418792ba801b45f1f663 [id=b1b956e2eb0740bab149e499b6707263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.454173 12895 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f666e1a8d214172bf5b752e9ec374bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:59.454501 12565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0c465b503b54e569fbf529da3ac316c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.454953 12896 tablet_service.cc:1511] Processing CreateTablet for tablet 839b20750eba4d8185bc7057e37da5fe (DEFAULT_TABLE table=default.Table_46c845de9a6f4e368a893c401a7f43d2 [id=68d798ba84c54907b518ffbb6bb1064e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.455194 12890 tablet_service.cc:1511] Processing CreateTablet for tablet 8d172eb62e144cc991370198c608ab58 (DEFAULT_TABLE table=default.Table_da82d46a08d745ddaebd2a7c79b1d4be [id=843400c90a554f839014213cd53c1379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.455994 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839b20750eba4d8185bc7057e37da5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.456553 12890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d172eb62e144cc991370198c608ab58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.461409 12738 tablet_service.cc:1558] Processing DeleteTablet for tablet 6f666e1a8d214172bf5b752e9ec374bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:59.470374 12895 tablet_service.cc:1511] Processing CreateTablet for tablet 636169e8ee334ac4898973c33735b763 (DEFAULT_TABLE table=default.Table_94d1edd7294d4dc38229784cab1912ea [id=02e8cd9af9cc4dc98eced0e23adc461f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.470858 12892 tablet_service.cc:1511] Processing CreateTablet for tablet 036b9a2a0d65468ab8ff19ba06ec72a0 (DEFAULT_TABLE table=default.Table_6787fdb6bfad44bc8eb2f6c7c0848f48 [id=dc9290fc992b47229a40c49469284183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.471459 12895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 636169e8ee334ac4898973c33735b763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.471547 12901 tablet_service.cc:1511] Processing CreateTablet for tablet a1d5deb0364542a7945e5675ca31557c (DEFAULT_TABLE table=default.Table_85695893a5b24687959b83a8360d1443 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.471621 12902 tablet_service.cc:1511] Processing CreateTablet for tablet a0c465b503b54e569fbf529da3ac316c (DEFAULT_TABLE table=default.Table_c7ed422235cf418792ba801b45f1f663 [id=b1b956e2eb0740bab149e499b6707263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.472549 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1d5deb0364542a7945e5675ca31557c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.472343 12894 tablet_service.cc:1511] Processing CreateTablet for tablet 83a78d487f0147fb95e8b946539ab3a6 (DEFAULT_TABLE table=default.Table_94fb4afa42c74fa1b7ec938574559b46 [id=ba86aa08529b4135a224b8d53ffc7ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.473587 12248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 382 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:58:59.474108 12894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a78d487f0147fb95e8b946539ab3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.475155 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0c465b503b54e569fbf529da3ac316c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.477164 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb9c1e977f942d9b6e3a00c3f4bc694 (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.478216 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb9c1e977f942d9b6e3a00c3f4bc694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.480321 12892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 036b9a2a0d65468ab8ff19ba06ec72a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.487044 12734 tablet_service.cc:1511] Processing CreateTablet for tablet 43a92aa343944eab9c937aeeac953649 (DEFAULT_TABLE table=default.Table_c360f50af2554b0984e0b30eb5bbd833 [id=0fafc84100524ebe8f48618a6c3b9234]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.489789 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 636169e8ee334ac4898973c33735b763 (DEFAULT_TABLE table=default.Table_94d1edd7294d4dc38229784cab1912ea [id=02e8cd9af9cc4dc98eced0e23adc461f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.492650 12564 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb9c1e977f942d9b6e3a00c3f4bc694 (DEFAULT_TABLE table=default.Table_2c324c2f21db4f39a507bf9b47ff489d [id=5984b520ca4f4ea29814caadf67f878e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.493288 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 036b9a2a0d65468ab8ff19ba06ec72a0 (DEFAULT_TABLE table=default.Table_6787fdb6bfad44bc8eb2f6c7c0848f48 [id=dc9290fc992b47229a40c49469284183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.495060 12558 tablet_service.cc:1511] Processing CreateTablet for tablet a1d5deb0364542a7945e5675ca31557c (DEFAULT_TABLE table=default.Table_85695893a5b24687959b83a8360d1443 [id=185449ae1c9744eba4df8bd0f9704abb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.497165 12557 tablet_service.cc:1511] Processing CreateTablet for tablet 9ed22219df2b42ad86a6d7917ee0d990 (DEFAULT_TABLE table=default.Table_174debbe544e47ccbcf8faef177fdd0b [id=1a610b5d22a14bd79b983901002df704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.252876 13915 raft_consensus.cc:493] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:59.500734 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 1c38a5aab5eb4404b783eaa5cc477c31 (DEFAULT_TABLE table=default.Table_8d453d8c87c949c6b28b98fdf69184d3 [id=7ac713532b6642159fcf7f2b06516cae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.501127 12555 tablet_service.cc:1511] Processing CreateTablet for tablet 83a78d487f0147fb95e8b946539ab3a6 (DEFAULT_TABLE table=default.Table_94fb4afa42c74fa1b7ec938574559b46 [id=ba86aa08529b4135a224b8d53ffc7ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.503114 12641 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:59.492658 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 636169e8ee334ac4898973c33735b763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.507895 12557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ed22219df2b42ad86a6d7917ee0d990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.507748 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 353fd263095d44ad8000a6a490565153 (DEFAULT_TABLE table=default.Table_4d7aa0cb1d464b23925eff429da34da9 [id=5a30c44bbb544ecba001c38c2619ecfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.509359 12564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb9c1e977f942d9b6e3a00c3f4bc694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.515492 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 307) took 1469 ms. Trace:
I20260926 19:58:59.515817 13569 rpcz_store.cc:276] 0926 19:58:58.046004 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:58.046197 (+ 193us) service_pool.cc:224] Handling call
0926 19:58:58.945867 (+899670us) ts_tablet_manager.cc:1151] Deleting tablet 89c8cbb097a644218e0ca033a6fdb4a2
0926 19:58:58.945932 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:59.195555 (+249623us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:59.299505 (+103950us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:59.515433 (+215928us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71214,"tablet-delete.queue_time_us":899038}
I20260926 19:58:59.517213 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 308) took 1466 ms. Trace:
I20260926 19:58:59.519032 13013 tablet_bootstrap.cc:492] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:59.518983 13569 rpcz_store.cc:276] 0926 19:58:58.050530 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:58.050662 (+ 132us) service_pool.cc:224] Handling call
0926 19:58:59.516877 (+1466215us) ts_tablet_manager.cc:1151] Deleting tablet e9c7b63db44742108dc132df5089e0d9
0926 19:58:59.516919 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:59.517163 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1465650}
I20260926 19:58:59.519555 13013 ts_tablet_manager.cc:1403] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.394s user 0.007s sys 0.006s
I20260926 19:58:59.530081 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 382 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:58:59.540485 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534
I20260926 19:58:59.543684 13739 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:58:59.545583 13926 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:58:59.545828 12555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a78d487f0147fb95e8b946539ab3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.548247 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 01d8740b03594023a8420c7a7121b80c
I20260926 19:58:59.549094 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 839b20750eba4d8185bc7057e37da5fe (DEFAULT_TABLE table=default.Table_46c845de9a6f4e368a893c401a7f43d2 [id=68d798ba84c54907b518ffbb6bb1064e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.549691 12392 catalog_manager.cc:6934] ReplaceTablet: tablet 01d8740b03594023a8420c7a7121b80c deleted and replaced by tablet 6c23ecdde3b04ce784f3f4bf0b2f0441
I20260926 19:58:59.552402 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 89c8cbb097a644218e0ca033a6fdb4a2 (table default.Table_27371e4eb0cd4fa48aaef20d2db04fcc [id=f5d5844ddbeb4542b5e7b02d22800604]) successfully deleted
W20260926 19:58:59.553260 12380 catalog_manager.cc:5043] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet e9c7b63db44742108dc132df5089e0d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9c7b63db44742108dc132df5089e0d9 already in progress: creating tablet
W20260926 19:58:59.554275 12380 catalog_manager.cc:5030] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): delete failed for tablet 06c5ef9e1bca427fb889cbb17f77f999 because the tablet was not found. No further retry: Not found: Tablet not found: 06c5ef9e1bca427fb889cbb17f77f999
I20260926 19:58:59.549058 12729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e776237ca24a32a8e73d2f0628df3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.562587 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 1c38a5aab5eb4404b783eaa5cc477c31 (DEFAULT_TABLE table=default.Table_8d453d8c87c949c6b28b98fdf69184d3 [id=7ac713532b6642159fcf7f2b06516cae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.565114 12554 tablet_service.cc:1511] Processing CreateTablet for tablet 4d43853cbe2e438c8588460d7b9c1660 (DEFAULT_TABLE table=default.Table_ef93987144814289b97049e7fbc68e1b [id=4249fab1d2fb4d189cf745e61dfe60f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.567715 12553 tablet_service.cc:1511] Processing CreateTablet for tablet 353fd263095d44ad8000a6a490565153 (DEFAULT_TABLE table=default.Table_4d7aa0cb1d464b23925eff429da34da9 [id=5a30c44bbb544ecba001c38c2619ecfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.571380 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_f1b2f134d88540dab02e8debcbdbc755"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:59.572902 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1b2f134d88540dab02e8debcbdbc755 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:58:59.574656 12392 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_174debbe544e47ccbcf8faef177fdd0b" } modify_external_catalogs: true
I20260926 19:58:59.575134 12392 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_174debbe544e47ccbcf8faef177fdd0b" } modify_external_catalogs: true
I20260926 19:58:59.577786 12730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad59d3cab654186ac2cbd3860980d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.578547 12388 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_156126e29ad847b1ad7d29c9b69aad51" } new_table_name: "default.Table_c86d8f83e71740b7954d3a18a6af3d87" modify_external_catalogs: true
I20260926 19:58:59.579494 13403 ts_tablet_manager.cc:1929] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:58:59.579841 13403 log.cc:1199] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/06c5ef9e1bca427fb889cbb17f77f999
I20260926 19:58:59.580513 13403 ts_tablet_manager.cc:1950] T 06c5ef9e1bca427fb889cbb17f77f999 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
W20260926 19:58:59.586216 12731 consensus_meta.cc:287] T cde094c41f8d4a4e8b2f34cea2825326 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.625s user 0.007s sys 0.000s
I20260926 19:58:59.587471 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:58 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:59.588074 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet 01d8740b03594023a8420c7a7121b80c with delete_type TABLET_DATA_DELETED (replaced by tablet 6c23ecdde3b04ce784f3f4bf0b2f0441) from {username='slave'} at 127.0.0.1:38310
I20260926 19:58:59.510861 13915 raft_consensus.cc:515] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.589898 13915 leader_election.cc:290] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:59.591708 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 342cc84ea5aa45259bf46068930eb921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.594260 12552 tablet_service.cc:1558] Processing DeleteTablet for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:58 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:59.594964 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3409a06964ba45069360f60a8c5d910d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.597666 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839b20750eba4d8185bc7057e37da5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.598331 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d019d9e2bab4fd3a92b921d972a4cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.599072 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353fd263095d44ad8000a6a490565153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.599668 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a92aa343944eab9c937aeeac953649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.635274 14002 ts_tablet_manager.cc:1929] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:59.640728 12735 tablet_service.cc:1511] Processing CreateTablet for tablet 4d43853cbe2e438c8588460d7b9c1660 (DEFAULT_TABLE table=default.Table_ef93987144814289b97049e7fbc68e1b [id=4249fab1d2fb4d189cf745e61dfe60f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:58:59.646618 14002 log.cc:1199] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/56fd4c88f9304f19883c3e5da8f937b5
I20260926 19:58:59.647502 14002 ts_tablet_manager.cc:1950] T 56fd4c88f9304f19883c3e5da8f937b5 P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:59.658193 14002 tablet_replica.cc:333] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:58:59.636992 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1d5deb0364542a7945e5675ca31557c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.637778 14001 raft_consensus.cc:493] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:59.666859 14001 raft_consensus.cc:515] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.668926 14001 leader_election.cc:290] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:59.633435 12552 tablet_service.cc:1558] Processing DeleteTablet for tablet 01d8740b03594023a8420c7a7121b80c with delete_type TABLET_DATA_DELETED (replaced by tablet 6c23ecdde3b04ce784f3f4bf0b2f0441) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:59.669914 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c38a5aab5eb4404b783eaa5cc477c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.662879 13901 raft_consensus.cc:493] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:59.670907 13901 raft_consensus.cc:515] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:58:59.667255 14002 raft_consensus.cc:2243] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:59.671788 14002 raft_consensus.cc:2272] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:59.672837 13901 leader_election.cc:290] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:58:59.673931 14002 ts_tablet_manager.cc:1916] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:59.636037 13013 raft_consensus.cc:359] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.676851 13013 raft_consensus.cc:385] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:59.677102 13013 raft_consensus.cc:740] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:59.678306 12553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353fd263095d44ad8000a6a490565153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.677691 13013 consensus_queue.cc:260] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.681054 13013 ts_tablet_manager.cc:1434] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.161s user 0.000s sys 0.004s
I20260926 19:58:59.681974 13013 tablet_bootstrap.cc:492] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:59.682041 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 036b9a2a0d65468ab8ff19ba06ec72a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.703263 12554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d43853cbe2e438c8588460d7b9c1660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.725973 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839b20750eba4d8185bc7057e37da5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.738788 13901 raft_consensus.cc:493] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:58:59.742238 13901 raft_consensus.cc:515] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.744904 13901 leader_election.cc:290] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:58:59.749954 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_019731a69622436792bb7a8512e0cb49" } modify_external_catalogs: true
I20260926 19:58:59.750442 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_019731a69622436792bb7a8512e0cb49" } modify_external_catalogs: true
I20260926 19:58:59.758486 12904 tablet_service.cc:1558] Processing DeleteTablet for tablet 01d8740b03594023a8420c7a7121b80c with delete_type TABLET_DATA_DELETED (replaced by tablet 6c23ecdde3b04ce784f3f4bf0b2f0441) from {username='slave'} at 127.0.0.1:33594
I20260926 19:58:59.774071 13768 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" instance_seqno: 1790452712125164) as {username='slave'} at 127.11.233.194:50819; Asking this server to re-register.
I20260926 19:58:59.790562 13013 tablet_bootstrap.cc:654] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:58:59.825074 13769 ts_manager.cc:194] Registered new tserver with Master: f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:58:59.828595 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 56fd4c88f9304f19883c3e5da8f937b5 (table default.Table_67c2bbc7541d4c2fb17c5ea6c2a33e8b [id=5639673ab1364e00a44bc7bce0428012]) successfully deleted
I20260926 19:58:59.842090 13768 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" instance_seqno: 1790452714684854) as {username='slave'} at 127.11.233.195:59223; Asking this server to re-register.
I20260926 19:58:59.846202 12970 heartbeater.cc:461] Registering TS with master...
I20260926 19:58:59.860047 13013 tablet_bootstrap.cc:492] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:58:59.860622 13013 ts_tablet_manager.cc:1403] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.179s user 0.008s sys 0.003s
I20260926 19:58:59.863608 13013 raft_consensus.cc:359] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.540246 13569 tablet_replica.cc:333] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:58:59.685789 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f59be98ceef4effa15baf487c2d1140" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:59.687608 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4d86fa6b40b4fc49c4fa67b63792831" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:59.897751 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9f7a2e5d1c41a8ac793a31696fb311" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:58:59.691923 13012 tablet_bootstrap.cc:492] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:58:59.906703 13012 ts_tablet_manager.cc:1403] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 1.002s user 0.022s sys 0.009s
I20260926 19:58:59.695822 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d172eb62e144cc991370198c608ab58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.908541 12722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 036b9a2a0d65468ab8ff19ba06ec72a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.910172 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc9f7a2e5d1c41a8ac793a31696fb311" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:58:59.922843 12389 master_service.cc:960] ReplaceTablet: received request to replace tablet f28394cd29c44af6b8c662f6c9e1165d from {username='slave'} at 127.0.0.1:45734
W20260926 19:58:59.865576 12695 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s user 0.000s sys 0.016s
W20260926 19:58:59.938951 12695 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s user 0.000s sys 0.016s
W20260926 19:58:59.899091 12521 leader_election.cc:343] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:59.904621 12519 leader_election.cc:343] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:58:59.942073 12519 leader_election.cc:343] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:59.942137 13569 raft_consensus.cc:2243] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:59.944458 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c38a5aab5eb4404b783eaa5cc477c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:58:59.945613 13013 raft_consensus.cc:385] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:58:59.945947 13013 raft_consensus.cc:740] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:58:59.946743 13013 consensus_queue.cc:260] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:58:59.934116 14002 ts_tablet_manager.cc:1929] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:58:59.956314 14002 log.cc:1199] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/6f666e1a8d214172bf5b752e9ec374bd
I20260926 19:58:59.957595 12392 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 9ed22219df2b42ad86a6d7917ee0d990
I20260926 19:58:59.960093 13013 ts_tablet_manager.cc:1434] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.099s user 0.001s sys 0.005s
I20260926 19:58:59.961212 13013 tablet_bootstrap.cc:492] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:58:59.966297 14002 ts_tablet_manager.cc:1950] T 6f666e1a8d214172bf5b752e9ec374bd P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:58:59.969590 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 6f666e1a8d214172bf5b752e9ec374bd (table default.Table_c7d455485dd14d60806f7a48b45ce98b [id=453a40bc60fd4eeaa2ffc14d887320ec]) successfully deleted
I20260926 19:58:59.970260 14002 tablet_replica.cc:333] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
W20260926 19:58:59.970595 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 already in progress: creating tablet
I20260926 19:58:59.971236 14002 raft_consensus.cc:2243] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:58:59.971764 14002 raft_consensus.cc:2272] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:58:59.974421 12519 leader_election.cc:343] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:58:59.974800 12519 leader_election.cc:304] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:58:59.975945 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d43853cbe2e438c8588460d7b9c1660. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:58:59.967379 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 751544us
I20260926 19:58:59.976382 14001 raft_consensus.cc:2749] T cc9f7a2e5d1c41a8ac793a31696fb311 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:58:59.976866 12695 leader_election.cc:343] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 59f7dc92330d400f93f427f3f79ad826
I20260926 19:58:59.976894 12557 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ed22219df2b42ad86a6d7917ee0d990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:59 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:58:59.977365 12695 leader_election.cc:304] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:58:59.979082 13569 pending_rounds.cc:70] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Trying to abort 1 pending ops.
I20260926 19:58:59.979375 13569 pending_rounds.cc:77] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694414915842048 op_type: NO_OP noop_request { }
I20260926 19:58:59.979770 13569 raft_consensus.cc:2889] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:58:59.980180 13569 raft_consensus.cc:2272] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:58:59.973599 14002 ts_tablet_manager.cc:1916] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:58:59.996604 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_93b04d48f59c40bc9c1ab2ca57e0bef4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:58:59.998438 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93b04d48f59c40bc9c1ab2ca57e0bef4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:00.016947 12388 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 734c7713a06449c190105c1478b62251 consensus state has no leader
I20260926 19:59:00.022225 12889 tablet_service.cc:1511] Processing CreateTablet for tablet 9ed22219df2b42ad86a6d7917ee0d990 (DEFAULT_TABLE table=default.Table_174debbe544e47ccbcf8faef177fdd0b [id=1a610b5d22a14bd79b983901002df704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.023226 12885 tablet_service.cc:1558] Processing DeleteTablet for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:58 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:00.026149 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_c3fe6fcf32d349ad852cc8a8b766c175"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.027590 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c3fe6fcf32d349ad852cc8a8b766c175 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.046002 12886 tablet_service.cc:1511] Processing CreateTablet for tablet 353fd263095d44ad8000a6a490565153 (DEFAULT_TABLE table=default.Table_4d7aa0cb1d464b23925eff429da34da9 [id=5a30c44bbb544ecba001c38c2619ecfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.049605 12887 tablet_service.cc:1511] Processing CreateTablet for tablet 4d43853cbe2e438c8588460d7b9c1660 (DEFAULT_TABLE table=default.Table_ef93987144814289b97049e7fbc68e1b [id=4249fab1d2fb4d189cf745e61dfe60f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.051622 13803 consensus_queue.cc:1048] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:00.056717 13013 tablet_bootstrap.cc:654] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:00.059682 12394 master_service.cc:960] ReplaceTablet: received request to replace tablet 71065b80482442699f4b207cb8f9d40c from {username='slave'} at 127.0.0.1:45734
I20260926 19:59:00.062283 13994 raft_consensus.cc:493] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:00.035422 12888 tablet_service.cc:1511] Processing CreateTablet for tablet 1c38a5aab5eb4404b783eaa5cc477c31 (DEFAULT_TABLE table=default.Table_8d453d8c87c949c6b28b98fdf69184d3 [id=7ac713532b6642159fcf7f2b06516cae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.065301 14002 ts_tablet_manager.cc:1929] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:00.077683 14002 log.cc:1199] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/01d8740b03594023a8420c7a7121b80c
I20260926 19:59:00.078508 14002 ts_tablet_manager.cc:1950] T 01d8740b03594023a8420c7a7121b80c P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:59:00.081091 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 01d8740b03594023a8420c7a7121b80c (table default.Table_b638b85037f24946b493b98629cab24e [id=b748bf49932b4e0db7de11beebb15173]) successfully deleted
W20260926 19:59:00.070456 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 96054.3us
I20260926 19:59:00.062750 13994 raft_consensus.cc:515] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.086156 12389 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet f28394cd29c44af6b8c662f6c9e1165d
I20260926 19:59:00.086978 13994 leader_election.cc:290] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:00.087433 12389 catalog_manager.cc:6934] ReplaceTablet: tablet f28394cd29c44af6b8c662f6c9e1165d deleted and replaced by tablet fc5fd284545346cebf6cdd94cddb9136
I20260926 19:59:00.097056 12556 tablet_service.cc:1558] Processing DeleteTablet for tablet f28394cd29c44af6b8c662f6c9e1165d with delete_type TABLET_DATA_DELETED (replaced by tablet fc5fd284545346cebf6cdd94cddb9136) from {username='slave'} at 127.0.0.1:37792
W20260926 19:59:00.063962 12896 consensus_meta.cc:287] T 839b20750eba4d8185bc7057e37da5fe P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent flushing consensus metadata: real 0.597s user 0.003s sys 0.005s
W20260926 19:59:00.104945 12890 consensus_meta.cc:287] T 8d172eb62e144cc991370198c608ab58 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent flushing consensus metadata: real 0.640s user 0.006s sys 0.005s
I20260926 19:59:00.114023 13986 catalog_manager.cc:1308] Loaded cluster ID: 74efea6f15be4c5bb4893708cb04c269
I20260926 19:59:00.114645 13986 catalog_manager.cc:1601] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: loading cluster ID for follower catalog manager: success
W20260926 19:59:00.127612 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 9ed22219df2b42ad86a6d7917ee0d990 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ed22219df2b42ad86a6d7917ee0d990 already in progress: creating tablet
W20260926 19:59:00.129333 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet f28394cd29c44af6b8c662f6c9e1165d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f28394cd29c44af6b8c662f6c9e1165d already in progress: creating tablet
I20260926 19:59:00.130990 13013 tablet_bootstrap.cc:492] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:59:00.132754 13013 ts_tablet_manager.cc:1403] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.172s user 0.003s sys 0.008s
I20260926 19:59:00.135367 13013 raft_consensus.cc:359] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.137916 13013 raft_consensus.cc:385] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:00.139600 13013 raft_consensus.cc:740] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:59:00.141362 13013 consensus_queue.cc:260] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.147176 12918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f59be98ceef4effa15baf487c2d1140" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:00.149724 12920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4d86fa6b40b4fc49c4fa67b63792831" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:00.153095 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7d0435f3564cb38b6cbca713009c2e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:00.158344 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_2627a2020d2043c1a7553b82baeeed59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.159835 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2627a2020d2043c1a7553b82baeeed59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.160590 12889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ed22219df2b42ad86a6d7917ee0d990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.160970 13013 ts_tablet_manager.cc:1434] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.028s user 0.002s sys 0.003s
I20260926 19:59:00.162235 13013 tablet_bootstrap.cc:492] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:59:00.164240 12394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_87cff927dd5d4f0d9d947b7a5629728c" } modify_external_catalogs: true
I20260926 19:59:00.164723 12394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_87cff927dd5d4f0d9d947b7a5629728c" } modify_external_catalogs: true
I20260926 19:59:00.179039 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 288) took 3064 ms. Trace:
I20260926 19:59:00.181157 13013 tablet_bootstrap.cc:654] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:00.181805 12891 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 300) took 1201 ms. Trace:
I20260926 19:59:00.182150 12891 rpcz_store.cc:276] 0926 19:58:58.980743 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:58.981914 (+ 1171us) service_pool.cc:224] Handling call
0926 19:58:59.144239 (+162325us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.144334 (+ 95us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:59.415179 (+270845us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:59.899162 (+483983us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:00.181752 (+282590us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:00.189020 12886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 353fd263095d44ad8000a6a490565153. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.193676 12888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c38a5aab5eb4404b783eaa5cc477c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.198948 12887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d43853cbe2e438c8588460d7b9c1660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.203327 13013 tablet_bootstrap.cc:492] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:59:00.203742 13013 ts_tablet_manager.cc:1403] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.042s user 0.010s sys 0.001s
I20260926 19:59:00.220726 12389 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_85695893a5b24687959b83a8360d1443" } new_table_name: "default.Table_7a74cde0204a4a97953baf0c9f7b1968" modify_external_catalogs: true
I20260926 19:59:00.222262 13986 catalog_manager.cc:1623] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: acquiring CA information for follower catalog manager: success
I20260926 19:59:00.219821 13013 raft_consensus.cc:359] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.224524 13013 raft_consensus.cc:385] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:00.224858 13013 raft_consensus.cc:740] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:59:00.225485 13013 consensus_queue.cc:260] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.236572 14017 raft_consensus.cc:493] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:00.237015 14017 raft_consensus.cc:515] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:59:00.237880 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 317303us
I20260926 19:59:00.238660 13989 consensus_queue.cc:1048] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:00.239054 14017 leader_election.cc:290] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
W20260926 19:59:00.245246 12694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260926 19:59:00.181027 13403 rpcz_store.cc:276] 0926 19:58:57.114432 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.114765 (+ 333us) service_pool.cc:224] Handling call
0926 19:58:59.134704 (+2019939us) ts_tablet_manager.cc:1151] Deleting tablet 06c5ef9e1bca427fb889cbb17f77f999
0926 19:58:59.134740 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:58:59.254339 (+119599us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:59.579354 (+325015us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.178968 (+599614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":337790,"tablet-delete.queue_time_us":2010535}
I20260926 19:59:00.253160 13403 tablet_replica.cc:333] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:59:00.263819 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ec974c84b69a46b3a8820e70e43e7927"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.265295 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec974c84b69a46b3a8820e70e43e7927 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:00.269575 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 74fe405570f8474fb7a1f841a783a90a (table default.Table_2246cb86fb8e49de90ee0350d6287a8c [id=2d2ae06595164e708751168edcdad523], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 74fe405570f8474fb7a1f841a783a90a consensus state has no leader
I20260926 19:59:00.246402 12740 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 312) took 1139 ms. Trace:
I20260926 19:59:00.272603 12740 rpcz_store.cc:276] 0926 19:58:59.106866 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.107307 (+ 441us) service_pool.cc:224] Handling call
0926 19:58:59.510204 (+402897us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.510292 (+ 88us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:59.920343 (+410051us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.008928 (+ 88585us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:00.246337 (+237409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:00.273399 14017 raft_consensus.cc:493] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:00.273896 14017 raft_consensus.cc:515] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.275271 14028 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:59:00.275776 14028 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
W20260926 19:59:00.277640 12695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 300782us
I20260926 19:59:00.279910 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "700d05149a934d5e8bec9cc41873ad3e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:59:00.280651 12594 raft_consensus.cc:2468] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:59:00.301240 13013 ts_tablet_manager.cc:1434] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.097s user 0.003s sys 0.005s
I20260926 19:59:00.306509 13403 raft_consensus.cc:2243] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:00.261778 13092 tablet_bootstrap.cc:492] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:59:00.307677 13092 ts_tablet_manager.cc:1403] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.973s user 0.017s sys 0.005s
I20260926 19:59:00.307476 12592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7d0435f3564cb38b6cbca713009c2e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
W20260926 19:59:00.290540 12858 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 50433.7us
I20260926 19:59:00.308153 12592 raft_consensus.cc:2468] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:59:00.313616 14017 leader_election.cc:290] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:59:00.316920 14017 raft_consensus.cc:493] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:00.317315 14017 raft_consensus.cc:515] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.319182 14017 leader_election.cc:290] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:59:00.325057 14028 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:59:00.325888 12251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 382 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:59:00.326049 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 382 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:59:00.327674 13739 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:59:00.334772 13013 tablet_bootstrap.cc:492] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:59:00.342326 12395 catalog_manager.cc:5838] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:59:00.343830 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:59:00.344640 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 9b2d4d45fdeb4802ac261b6c8de91c3c (table default.Table_abfd9bc64fc246afb766ca3bd56518e5 [id=c010c3a0beb1406983b45e32c3e5640d]). Expected version 1 got 0
I20260926 19:59:00.345675 12395 catalog_manager.cc:5838] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194). New cstate: current_term: 1 leader_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } health_report { overall_health: HEALTHY } } }
I20260926 19:59:00.346796 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49]). Expected version 1 got 0
I20260926 19:59:00.347464 12395 catalog_manager.cc:5938] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461) does not have the latest schema for tablet 552d11cb23144691809c554c125cad0e (table default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504]). Expected version 1 got 0
I20260926 19:59:00.352470 13986 catalog_manager.cc:1651] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:59:00.352849 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet f28394cd29c44af6b8c662f6c9e1165d with delete_type TABLET_DATA_DELETED (replaced by tablet fc5fd284545346cebf6cdd94cddb9136) from {username='slave'} at 127.0.0.1:38310
W20260926 19:59:00.353569 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 95258.3us
I20260926 19:59:00.357816 13569 ts_tablet_manager.cc:1916] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:00.358295 14028 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:00.363286 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 3488f0076b0f4ef19f7cf23a93cd20e3
W20260926 19:59:00.363732 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 347969us
W20260926 19:59:00.366046 12521 leader_election.cc:343] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:00.366452 12521 leader_election.cc:304] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:59:00.370252 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet 3488f0076b0f4ef19f7cf23a93cd20e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:59:00.370412 12521 leader_election.cc:343] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:00.370820 12521 leader_election.cc:304] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:59:00.373227 13901 raft_consensus.cc:2749] T c4d86fa6b40b4fc49c4fa67b63792831 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:00.372829 14001 raft_consensus.cc:2749] T 8f59be98ceef4effa15baf487c2d1140 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:00.374274 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 31f10eb547ab4ab09e5b1e2a4f52bfcb (DEFAULT_TABLE table=default.Table_640b34d9f5ac410a82cf9c2ba23cd525 [id=7da8c8c422224467b70a086c631503a1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:00.375322 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 3488f0076b0f4ef19f7cf23a93cd20e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3488f0076b0f4ef19f7cf23a93cd20e3 already in progress: creating tablet
I20260926 19:59:00.377637 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f10eb547ab4ab09e5b1e2a4f52bfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.391953 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 6a85852c08304e0fbbbcbb2c0d24b27e
I20260926 19:59:00.391965 12740 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ed22219df2b42ad86a6d7917ee0d990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:59 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:00.353143 12725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ed22219df2b42ad86a6d7917ee0d990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.398102 13962 consensus_queue.cc:1048] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:00.351023 13840 raft_consensus.cc:2749] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:00.412405 12739 tablet_service.cc:1558] Processing DeleteTablet for tablet 3488f0076b0f4ef19f7cf23a93cd20e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:00.412895 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a85852c08304e0fbbbcbb2c0d24b27e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:59 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:00.413117 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ca451e5139456b9deb41a4fc635c87" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:00.413818 12760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54553d20d48148ae88fa4cc2ce021779" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:00.414516 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 31f10eb547ab4ab09e5b1e2a4f52bfcb (DEFAULT_TABLE table=default.Table_640b34d9f5ac410a82cf9c2ba23cd525 [id=7da8c8c422224467b70a086c631503a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.392995 13012 raft_consensus.cc:359] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.415721 13012 raft_consensus.cc:385] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:00.416039 13012 raft_consensus.cc:740] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:59:00.416996 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fe405570f8474fb7a1f841a783a90a" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:00.418777 13960 raft_consensus.cc:493] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:00.419272 13960 raft_consensus.cc:515] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.423395 13013 tablet_bootstrap.cc:654] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
W20260926 19:59:00.423985 12519 leader_election.cc:343] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:59:00.424578 12519 leader_election.cc:343] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:59:00.425094 12519 leader_election.cc:343] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:00.421394 13960 leader_election.cc:290] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:00.307080 13403 pending_rounds.cc:70] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Trying to abort 1 pending ops.
I20260926 19:59:00.437619 13403 pending_rounds.cc:77] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694414667137024 op_type: NO_OP noop_request { }
I20260926 19:59:00.454766 12567 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a85852c08304e0fbbbcbb2c0d24b27e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:59 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:00.455236 12556 tablet_service.cc:1511] Processing CreateTablet for tablet 2315702ccbe74c0197a834f1f59ad0df (DEFAULT_TABLE table=default.Table_b8021096ec5e4b29ada2934d6401a48d [id=0bad243ae1ea4cf2a5274e72ec863be5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:00.459201 12381 catalog_manager.cc:5043] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 6a85852c08304e0fbbbcbb2c0d24b27e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a85852c08304e0fbbbcbb2c0d24b27e already in progress: creating tablet
I20260926 19:59:00.464139 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f10eb547ab4ab09e5b1e2a4f52bfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.479936 12394 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_8ec529fa6fe04ad1b44a473e8610375a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:00.480332 12567 tablet_service.cc:1511] Processing CreateTablet for tablet ab4b47bb2a7f4b2a855cbb299ef665de (DEFAULT_TABLE table=default.Table_b840208ddba74ea084e0e59efccb56f5 [id=635811e25a6d495b829d241ae4e35af7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:00.481432 12394 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ec529fa6fe04ad1b44a473e8610375a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.482744 12392 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_1ef033f88cc24cfb911236e339e7ea2d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.484575 12392 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ef033f88cc24cfb911236e339e7ea2d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.485381 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_b68ea29b70b74dfe8183c6af78288a8b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.486749 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b68ea29b70b74dfe8183c6af78288a8b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.495361 12728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a78d487f0147fb95e8b946539ab3a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.499778 12739 tablet_service.cc:1511] Processing CreateTablet for tablet 2315702ccbe74c0197a834f1f59ad0df (DEFAULT_TABLE table=default.Table_b8021096ec5e4b29ada2934d6401a48d [id=0bad243ae1ea4cf2a5274e72ec863be5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.501948 13403 raft_consensus.cc:2889] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:59:00.503340 13403 raft_consensus.cc:2272] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:00.504325 12388 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49]). Expected version 1 got 0
I20260926 19:59:00.504984 12388 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
W20260926 19:59:00.505963 12389 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1d5deb0364542a7945e5675ca31557c consensus state has no leader
I20260926 19:59:00.506924 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_51e292b5ade74377a0aa791835ab0ebe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.508312 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51e292b5ade74377a0aa791835ab0ebe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.513865 12732 tablet_service.cc:1511] Processing CreateTablet for tablet ab4b47bb2a7f4b2a855cbb299ef665de (DEFAULT_TABLE table=default.Table_b840208ddba74ea084e0e59efccb56f5 [id=635811e25a6d495b829d241ae4e35af7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.515578 12724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 636169e8ee334ac4898973c33735b763. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.516266 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab4b47bb2a7f4b2a855cbb299ef665de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.516856 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2315702ccbe74c0197a834f1f59ad0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.518260 12556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2315702ccbe74c0197a834f1f59ad0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.519340 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab4b47bb2a7f4b2a855cbb299ef665de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.519658 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_abeab641cddf49ca82c394e6e2072af4 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 3 got 0
I20260926 19:59:00.520654 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet cc9f7a2e5d1c41a8ac793a31696fb311 (table default.Table_24dedc75d9844c41976c575db604e1d7 [id=87b4a0b0de7f410a8843a8c5e7ddee9c]). Expected version 1 got 0
W20260926 19:59:00.445009 12758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s user 0.005s sys 0.001s
I20260926 19:59:00.521732 12390 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 986a28107bd54199bf2cda96978150c3 (table default.Table_98e4f6017318449bbfcf847f5d8b51ee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]). Expected version 2 got 0
I20260926 19:59:00.523388 12393 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:45734 (request call id 402) took 1524 ms. Trace:
I20260926 19:59:00.524003 12727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb9c1e977f942d9b6e3a00c3f4bc694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.530684 13012 consensus_queue.cc:260] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:59:00.532196 12387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 70) took 1912 ms (client timeout 1000 ms). Trace:
I20260926 19:59:00.532464 14017 raft_consensus.cc:493] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:00.532960 14017 raft_consensus.cc:515] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:00.535226 14017 leader_election.cc:290] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:59:00.523703 12393 rpcz_store.cc:276] 0926 19:58:58.998928 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.298618 (+299690us) service_pool.cc:224] Handling call
0926 19:58:59.300120 (+ 1502us) catalog_manager.cc:2432] Looking up, locking, and authorizing table
0926 19:58:59.300996 (+ 876us) catalog_manager.cc:3764] Apply alter schema
0926 19:58:59.301652 (+ 656us) catalog_manager.cc:3796] Acquired catalog manager lock
0926 19:58:59.301826 (+ 174us) catalog_manager.cc:3954] Updating metadata on disk
0926 19:58:59.693104 (+391278us) catalog_manager.cc:3985] Committing alterations to in-memory state
0926 19:59:00.523333 (+830229us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:59.617468 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:59.634793 (+ 17325us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:59.634817 (+ 24us) write_op.cc:435] Acquired schema lock
0926 19:58:59.634830 (+ 13us) tablet.cc:662] Decoding operations
0926 19:58:59.635075 (+ 245us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:59.635113 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:59.635134 (+ 21us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:59.635263 (+ 129us) tablet.cc:701] Row locks acquired
0926 19:58:59.635275 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:59.644410 (+ 9135us) write_op.cc:270] Start()
0926 19:58:59.644505 (+ 95us) write_op.cc:276] Timestamp: P: 1790452739644336 usec, L: 0
0926 19:58:59.644522 (+ 17us) op_driver.cc:348] REPLICATION: starting
0926 19:58:59.645047 (+ 525us) log.cc:844] Serialized 405 byte log entry
0926 19:58:59.660216 (+ 15169us) op_driver.cc:464] REPLICATION: finished
0926 19:58:59.664964 (+ 4748us) write_op.cc:301] APPLY: starting
0926 19:58:59.665017 (+ 53us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:59.665098 (+ 81us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:59.665193 (+ 95us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:59.665361 (+ 168us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:59.666248 (+ 887us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:59.692446 (+ 26198us) write_op.cc:312] APPLY: finished
0926 19:58:59.692650 (+ 204us) log.cc:844] Serialized 23 byte log entry
0926 19:58:59.692796 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:59.692860 (+ 64us) write_op.cc:454] Released schema lock
0926 19:58:59.692890 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:59.693460 (+ 570us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:59.693486 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4596,"apply.run_cpu_time_us":1362,"apply.run_wall_time_us":28451,"num_ops":1,"prepare.queue_time_us":311752,"prepare.run_cpu_time_us":1962,"prepare.run_wall_time_us":27860,"raft.queue_time_us":84,"raft.run_cpu_time_us":687,"raft.run_wall_time_us":688,"replication_time_us":15606}]]}
W20260926 19:59:00.532603 12387 rpcz_store.cc:269] 0926 19:58:58.620100 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:58.620227 (+ 127us) service_pool.cc:224] Handling call
0926 19:58:58.737739 (+117512us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:59:00.532120 (+1794381us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:59.647186 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:58:59.647512 (+ 326us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:58:59.647533 (+ 21us) write_op.cc:435] Acquired schema lock
0926 19:58:59.647545 (+ 12us) tablet.cc:662] Decoding operations
0926 19:58:59.647752 (+ 207us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:58:59.647782 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0926 19:58:59.647802 (+ 20us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:58:59.647934 (+ 132us) tablet.cc:701] Row locks acquired
0926 19:58:59.647946 (+ 12us) write_op.cc:260] PREPARE: finished
0926 19:58:59.648034 (+ 88us) write_op.cc:270] Start()
0926 19:58:59.648102 (+ 68us) write_op.cc:276] Timestamp: P: 1790452739647983 usec, L: 0
0926 19:58:59.648117 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:58:59.648523 (+ 406us) log.cc:844] Serialized 536 byte log entry
0926 19:58:59.733025 (+ 84502us) op_driver.cc:464] REPLICATION: finished
0926 19:58:59.735033 (+ 2008us) write_op.cc:301] APPLY: starting
0926 19:58:59.735084 (+ 51us) tablet.cc:1394] starting BulkCheckPresence
0926 19:58:59.735173 (+ 89us) tablet.cc:1397] finished BulkCheckPresence
0926 19:58:59.735254 (+ 81us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:58:59.735449 (+ 195us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:58:59.735570 (+ 121us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:58:59.735612 (+ 42us) write_op.cc:312] APPLY: finished
0926 19:58:59.735769 (+ 157us) log.cc:844] Serialized 23 byte log entry
0926 19:58:59.735890 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:59.735969 (+ 79us) write_op.cc:454] Released schema lock
0926 19:58:59.736001 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0926 19:58:59.736200 (+ 199us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:58:59.736230 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"spinlock_wait_cycles":243009280,"child_traces":[["op",{"apply.queue_time_us":1892,"apply.run_cpu_time_us":1145,"apply.run_wall_time_us":1145,"num_ops":1,"prepare.queue_time_us":307106,"prepare.run_cpu_time_us":1489,"prepare.run_wall_time_us":1488,"replication_time_us":84838}]]}
W20260926 19:59:00.529373 12391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 62) took 2038 ms (client timeout 1000 ms). Trace:
W20260926 19:59:00.542418 12391 rpcz_store.cc:269] 0926 19:58:58.490830 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:58.490960 (+ 130us) service_pool.cc:224] Handling call
0926 19:59:00.529308 (+2038348us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":6}
W20260926 19:59:00.544510 14033 log.cc:927] Time spent T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260926 19:59:00.548247 12557 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:00.548851 13012 ts_tablet_manager.cc:1434] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.642s user 0.002s sys 0.005s
I20260926 19:59:00.548943 12553 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 9599f427a64f40d7b185b23a849b46f3) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:00.549813 14002 tablet_replica.cc:333] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
W20260926 19:59:00.549801 13993 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s user 0.025s sys 0.029s
W20260926 19:59:00.550314 13993 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s user 0.025s sys 0.029s
I20260926 19:59:00.550618 14002 raft_consensus.cc:2243] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:00.551098 14002 raft_consensus.cc:2272] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:00.552516 13013 tablet_bootstrap.cc:492] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:59:00.556259 14002 ts_tablet_manager.cc:1916] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:00.561744 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3066a65ce44b4387a6f96124d9f476fb" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:00.562345 12553 tablet_service.cc:1511] Processing CreateTablet for tablet 1a5b12bdb6f346d99fd27c57fcd1064f (DEFAULT_TABLE table=default.Table_2aae6d0b16cf43b8ade8e77279c797b8 [id=79430eea271b4a3381b5fc56600210f8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:00.564000 12730 consensus_meta.cc:287] T 9ad59d3cab654186ac2cbd3860980d01 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.943s user 0.019s sys 0.000s
I20260926 19:59:00.565034 12730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 317) took 1325 ms. Trace:
W20260926 19:59:00.549190 13012 ts_tablet_manager.cc:1467] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9: Tablet startup took 642ms
W20260926 19:59:00.567453 12519 leader_election.cc:343] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:00.582484 12553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5b12bdb6f346d99fd27c57fcd1064f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.596405 12557 tablet_service.cc:1511] Processing CreateTablet for tablet 2cc8bbccd0bf403db22e3791b2fb854d (DEFAULT_TABLE table=default.Table_d6d28646357a4b26ba16bbb1a7eb01d9 [id=c4265098bdb34d0ab60545dd2293ba84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.553455 13013 ts_tablet_manager.cc:1403] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 0.219s user 0.010s sys 0.002s
W20260926 19:59:00.602362 12737 consensus_meta.cc:287] T 353fd263095d44ad8000a6a490565153 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.653s user 0.008s sys 0.000s
I20260926 19:59:00.602857 12554 tablet_service.cc:1558] Processing DeleteTablet for tablet 700d05149a934d5e8bec9cc41873ad3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:00.605634 12554 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f59be98ceef4effa15baf487c2d1140 with delete_type TABLET_DATA_DELETED (replaced by tablet 9411d3bca81641a4b3dfaffb2dc2bb51) from {username='slave'} at 127.0.0.1:37792
W20260926 19:59:00.614151 12395 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.194:33049 (request call id 58) took 1042 ms (client timeout 1000 ms). Trace:
I20260926 19:59:00.617827 14027 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20260926 19:59:00.618160 12390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.193:50225 (request call id 63) took 1112 ms (client timeout 1000 ms). Trace:
W20260926 19:59:00.620213 12390 rpcz_store.cc:269] 0926 19:58:59.506161 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.506296 (+ 135us) service_pool.cc:224] Handling call
0926 19:58:59.581843 (+ 75547us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:59:00.618093 (+1036250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35024,"reported_tablets":11,"spinlock_wait_cycles":149744000}
I20260926 19:59:00.626857 12554 tablet_service.cc:1511] Processing CreateTablet for tablet 3972c0cc04894010ba6aac0586d60c5b (DEFAULT_TABLE table=default.Table_a9722bfab4444bfabadaa2118c2da3b9 [id=089b4f87dcdf4a5e9dc035b3bdbcd57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.619549 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet 01d8740b03594023a8420c7a7121b80c with delete_type TABLET_DATA_DELETED (replaced by tablet 6c23ecdde3b04ce784f3f4bf0b2f0441) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:00.627897 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74fe405570f8474fb7a1f841a783a90a" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:00.630798 12568 tablet_service.cc:1511] Processing CreateTablet for tablet 8dbc482325c743e9964b5258b5cb620f (DEFAULT_TABLE table=default.Table_d39ede314be3449d96bbf00b8c9be90b [id=4a7273d6bbbd47568cc1c107b021161b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:00.619997 12395 rpcz_store.cc:269] 0926 19:58:59.572016 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.612586 (+ 40570us) service_pool.cc:224] Handling call
0926 19:58:59.620438 (+ 7852us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:59:00.614085 (+993647us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:59:00.414149 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:59:00.414577 (+ 428us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:59:00.414602 (+ 25us) write_op.cc:435] Acquired schema lock
0926 19:59:00.414615 (+ 13us) tablet.cc:662] Decoding operations
0926 19:59:00.414933 (+ 318us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:59:00.414976 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0926 19:59:00.415000 (+ 24us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:59:00.415258 (+ 258us) tablet.cc:701] Row locks acquired
0926 19:59:00.415272 (+ 14us) write_op.cc:260] PREPARE: finished
0926 19:59:00.415382 (+ 110us) write_op.cc:270] Start()
0926 19:59:00.415474 (+ 92us) write_op.cc:276] Timestamp: P: 1790452740415338 usec, L: 0
0926 19:59:00.415489 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:59:00.415954 (+ 465us) log.cc:844] Serialized 919 byte log entry
0926 19:59:00.470910 (+ 54956us) op_driver.cc:464] REPLICATION: finished
0926 19:59:00.471327 (+ 417us) write_op.cc:301] APPLY: starting
0926 19:59:00.471378 (+ 51us) tablet.cc:1394] starting BulkCheckPresence
0926 19:59:00.471478 (+ 100us) tablet.cc:1397] finished BulkCheckPresence
0926 19:59:00.471576 (+ 98us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:59:00.471895 (+ 319us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:59:00.472041 (+ 146us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0926 19:59:00.472081 (+ 40us) write_op.cc:312] APPLY: finished
0926 19:59:00.472248 (+ 167us) log.cc:844] Serialized 29 byte log entry
0926 19:59:00.472379 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:59:00.472445 (+ 66us) write_op.cc:454] Released schema lock
0926 19:59:00.472474 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0926 19:59:00.482229 (+ 9755us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:59:00.482288 (+ 59us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20046,"reported_tablets":21,"spinlock_wait_cycles":6470912,"child_traces":[["op",{"apply.queue_time_us":263,"apply.run_cpu_time_us":1362,"apply.run_wall_time_us":1362,"num_ops":2,"prepare.queue_time_us":64756,"prepare.run_cpu_time_us":6848,"prepare.run_wall_time_us":68000,"raft.queue_time_us":36813,"raft.run_cpu_time_us":671,"raft.run_wall_time_us":671,"replication_time_us":55335}]]}
I20260926 19:59:00.650079 13092 raft_consensus.cc:359] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:00.624326 12391 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_64c355e960994a9197d1f606cf6017a3 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 4 got 0
I20260926 19:59:00.650880 13092 raft_consensus.cc:385] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:00.651223 13092 raft_consensus.cc:740] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:59:00.651467 12391 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet cc9f7a2e5d1c41a8ac793a31696fb311 (table default.Table_24dedc75d9844c41976c575db604e1d7 [id=87b4a0b0de7f410a8843a8c5e7ddee9c]). Expected version 1 got 0
I20260926 19:59:00.651958 13092 consensus_queue.cc:260] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:00.652475 12391 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 986a28107bd54199bf2cda96978150c3 (table default.Table_98e4f6017318449bbfcf847f5d8b51ee [id=948f2bdffbc84b0ebf0f88e273f0a1fe]). Expected version 2 got 0
I20260926 19:59:00.653275 12391 catalog_manager.cc:5938] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439) does not have the latest schema for tablet 74fe405570f8474fb7a1f841a783a90a (table default.Table_2246cb86fb8e49de90ee0350d6287a8c [id=2d2ae06595164e708751168edcdad523]). Expected version 1 got 0
I20260926 19:59:00.600559 13013 raft_consensus.cc:359] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:00.653846 13013 raft_consensus.cc:385] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:00.654212 13013 raft_consensus.cc:740] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
I20260926 19:59:00.662964 12894 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 315) took 1192 ms. Trace:
I20260926 19:59:00.663425 12894 rpcz_store.cc:276] 0926 19:58:59.470377 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.471752 (+ 1375us) service_pool.cc:224] Handling call
0926 19:58:59.473469 (+ 1717us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.473779 (+ 310us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:00.096907 (+623128us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.592970 (+496063us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:00.662884 (+ 69914us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:00.666270 12558 tablet_service.cc:1511] Processing CreateTablet for tablet 57643cb928554e70a22c4b213411e0b2 (DEFAULT_TABLE table=default.Table_5284d05a66f641f1858d0b6dbf51ae4b [id=765c307ad69a4fb2a6befc0c55508351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.680902 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54553d20d48148ae88fa4cc2ce021779" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:00.680965 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14ca451e5139456b9deb41a4fc635c87" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:00.683804 12559 tablet_service.cc:1558] Processing DeleteTablet for tablet 54553d20d48148ae88fa4cc2ce021779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:57 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:00.685928 12559 tablet_service.cc:1511] Processing CreateTablet for tablet f6630cb35e584d488beb2b3f27355c19 (DEFAULT_TABLE table=default.Table_eb4069f7356a45b7bbbd58b430343d29 [id=16379e7ded6340149710638b5d04efc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.687009 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet 3066a65ce44b4387a6f96124d9f476fb with delete_type TABLET_DATA_DELETED (replaced by tablet f5b4b81f11254306b56c234f22b86ea5) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:00.693789 12395 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3024aa79f7c748b9aceef193e0602546" } modify_external_catalogs: true
I20260926 19:59:00.694306 12395 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3024aa79f7c748b9aceef193e0602546" } modify_external_catalogs: true
I20260926 19:59:00.713285 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_07843c7a37754743a74a6b3b822060a6" } new_table_name: "default.Table_8d7d5e4e1eae401aaab54c8a59c00e23" modify_external_catalogs: true
I20260926 19:59:00.671015 13013 consensus_queue.cc:260] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:00.718998 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 06c5ef9e1bca427fb889cbb17f77f999 (table default.Table_06d7f301c3cb4d0e8d374df103fab2f2 [id=ab940466309d4545b5a3f15ecc020742]) successfully deleted
I20260926 19:59:00.720100 13013 ts_tablet_manager.cc:1434] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee: Time spent starting tablet: real 0.122s user 0.005s sys 0.001s
I20260926 19:59:00.722461 13013 tablet_bootstrap.cc:492] T fe567f5d4c8c41ff9d681ca9ad3970e8 P f6e9d6439fd54b28925a68a6145d3dee: Bootstrap starting.
I20260926 19:59:00.730360 12557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cc8bbccd0bf403db22e3791b2fb854d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.730198 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ff2fc79b85024246be7c9dfd4c602925"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:00.731717 12554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3972c0cc04894010ba6aac0586d60c5b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:00.732136 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff2fc79b85024246be7c9dfd4c602925 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.732905 12568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dbc482325c743e9964b5258b5cb620f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.733791 12558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57643cb928554e70a22c4b213411e0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.734963 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6630cb35e584d488beb2b3f27355c19. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:00.737202 12390 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cb5e54722cfe43e5ada9feafaaa23964 consensus state has no leader
I20260926 19:59:00.739579 12395 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet e27c5b06d33c43ab9a483e1fc9b18234
I20260926 19:59:00.742924 12562 tablet_service.cc:1558] Processing DeleteTablet for tablet e27c5b06d33c43ab9a483e1fc9b18234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:00.743886 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_4ae8526c5c1f43859924b2e27dbbc75a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.745251 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ae8526c5c1f43859924b2e27dbbc75a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.748553 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_de6c58034b1949dfa5e5f83bd07db6d7" } new_table_name: "default.Table_8a7885640bb644a0abfa68ec902c48ed" modify_external_catalogs: true
I20260926 19:59:00.772002 12391 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_e4d93f2b15784ea8b7c7bdd2522017e5" } modify_external_catalogs: true
I20260926 19:59:00.773056 12391 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_e4d93f2b15784ea8b7c7bdd2522017e5" } modify_external_catalogs: true
W20260926 19:59:00.786619 12390 catalog_manager.cc:6106] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 81918e7e00f84a0ab8e19963a9d20332 consensus state has no leader
I20260926 19:59:00.795967 12567 tablet_service.cc:1511] Processing CreateTablet for tablet 708609d227e3431eb932c79289386afb (DEFAULT_TABLE table=default.Table_ffee871354884e4a882a0945a6a111c4 [id=36eb5997553f442395e21cb4693f1b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.803303 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 708609d227e3431eb932c79289386afb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:00.806380 12736 consensus_meta.cc:287] T 342cc84ea5aa45259bf46068930eb921 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.837s user 0.008s sys 0.000s
I20260926 19:59:00.808768 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 658c75b17f13420f8207f87a18eb9359 (DEFAULT_TABLE table=default.Table_c8b681e9cf4b4c26a3ea0fb2f83c0cd6 [id=a56e39982bb24f56a1348ef03eaf0510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.809860 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 658c75b17f13420f8207f87a18eb9359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.811885 12390 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_67a242744ca1463daaee9695a204dadd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.813346 12390 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67a242744ca1463daaee9695a204dadd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.816139 12733 tablet_service.cc:1558] Processing DeleteTablet for tablet 89c8cbb097a644218e0ca033a6fdb4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:00.817077 12737 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 335) took 1393 ms. Trace:
I20260926 19:59:00.819005 12737 rpcz_store.cc:276] 0926 19:58:59.424091 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.507172 (+ 83081us) service_pool.cc:224] Handling call
0926 19:58:59.508306 (+ 1134us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.508374 (+ 68us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:59.948803 (+440429us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.816598 (+867795us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:00.817022 (+ 424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:00.819862 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_1e0638a7221d47b19343a23c178452cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.821262 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e0638a7221d47b19343a23c178452cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.865654 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3066a65ce44b4387a6f96124d9f476fb" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:00.868523 12562 tablet_service.cc:1511] Processing CreateTablet for tablet 34c89822201b44d1bee654ee75f8c4a7 (DEFAULT_TABLE table=default.Table_c341da116f104491a6da32f46b955ecd [id=66c76c7dc6134a3e876cc48d9e663f74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.870436 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c89822201b44d1bee654ee75f8c4a7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:00.871544 12521 leader_election.cc:343] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:00.871990 12521 leader_election.cc:304] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
W20260926 19:59:00.873144 12521 leader_election.cc:343] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:00.874302 12521 leader_election.cc:304] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
W20260926 19:59:00.876094 12521 leader_election.cc:343] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:00.830842 12741 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 320) took 1483 ms. Trace:
I20260926 19:59:00.876681 12521 leader_election.cc:304] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:59:00.881753 14015 raft_consensus.cc:493] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:00.887607 12899 tablet_service.cc:1558] Processing DeleteTablet for tablet f28394cd29c44af6b8c662f6c9e1165d with delete_type TABLET_DATA_DELETED (replaced by tablet fc5fd284545346cebf6cdd94cddb9136) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:00.890147 12894 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ed22219df2b42ad86a6d7917ee0d990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:59 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:00.890295 13403 ts_tablet_manager.cc:1916] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:00.568285 12726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1d5deb0364542a7945e5675ca31557c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:00.568650 12758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s user 0.007s sys 0.006s
I20260926 19:59:00.901337 12391 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet dbd3a49039354c28b8faf1fd052210ee
I20260926 19:59:00.901880 12732 tablet_service.cc:1558] Processing DeleteTablet for tablet 700d05149a934d5e8bec9cc41873ad3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:00.877013 12741 rpcz_store.cc:276] 0926 19:58:59.347045 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.347181 (+ 136us) service_pool.cc:224] Handling call
0926 19:58:59.509704 (+162523us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.509785 (+ 81us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:59.983051 (+473266us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.814864 (+831813us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:00.830743 (+ 15879us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260926 19:59:00.905944 12695 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s user 0.017s sys 0.014s
I20260926 19:59:00.906184 13092 ts_tablet_manager.cc:1434] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.598s user 0.007s sys 0.000s
W20260926 19:59:00.906471 13092 ts_tablet_manager.cc:1467] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Tablet startup took 598ms
W20260926 19:59:00.863232 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 472215us
W20260926 19:59:00.906246 12695 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s user 0.017s sys 0.014s
W20260926 19:59:00.566440 13012 ts_tablet_manager.cc:1469] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9: Trace:
0926 19:58:45.467394 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:45.468564 (+ 1170us) service_pool.cc:224] Handling call
0926 19:58:45.470377 (+ 1813us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:45.470455 (+ 78us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:45.485737 (+ 15282us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:45.654204 (+168467us) tablet_replica.cc:206] Creating consensus instance
0926 19:58:45.654549 (+ 345us) inbound_call.cc:177] Queueing success response
0926 19:58:58.904097 (+13249548us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0926 19:58:59.907031 (+1002934us) ts_tablet_manager.cc:1435] Starting tablet replica
0926 19:58:59.907876 (+ 845us) tablet_replica.cc:261] Starting instrumentation
0926 19:58:59.969980 (+ 62104us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":42845,"raft.queue_time_us":144,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":276188,"tablet-open.queue_time_us":13249616}
I20260926 19:59:00.917371 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 3972c0cc04894010ba6aac0586d60c5b (DEFAULT_TABLE table=default.Table_a9722bfab4444bfabadaa2118c2da3b9 [id=089b4f87dcdf4a5e9dc035b3bdbcd57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.919031 12758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.11.233.195:39815 (request call id 139) took 1502 ms. Trace:
I20260926 19:59:00.920286 13403 ts_tablet_manager.cc:1929] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:00.920660 13403 log.cc:1199] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/89c8cbb097a644218e0ca033a6fdb4a2
I20260926 19:59:00.919414 12758 rpcz_store.cc:276] 0926 19:58:59.416308 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.426329 (+ 10021us) service_pool.cc:224] Handling call
0926 19:58:59.692602 (+266273us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de606db 00007f2c9de60761 00007f2c9de22570 00007f2c9de0cb69 00007f2c9de0c612 00007f2c9c488d34
0926 19:58:59.692760 (+ 158us) raft_consensus.cc:1471] Updating replica for 1 ops
0926 19:58:59.692997 (+ 237us) raft_consensus.cc:1534] Early marking committed up to index 0
0926 19:58:59.693022 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0926 19:58:59.693305 (+ 283us) alter_schema_op.cc:128] START. Timestamp: 1790452739282995
0926 19:58:59.694010 (+ 705us) log.cc:844] Serialized 173 byte log entry
0926 19:59:00.905696 (+1211686us) raft_consensus.cc:1645] Marking committed up to 3
0926 19:59:00.907432 (+ 1736us) raft_consensus.cc:1697] Filling consensus response to leader.
0926 19:59:00.907539 (+ 107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0926 19:59:00.907593 (+ 54us) raft_consensus.cc:1685] finished
0926 19:59:00.918787 (+ 11194us) raft_consensus.cc:1691] UpdateReplicas() finished
0926 19:59:00.918967 (+ 180us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:59.882764 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0926 19:58:59.883137 (+ 373us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0926 19:58:59.883169 (+ 32us) alter_schema_op.cc:66] Acquired schema lock
0926 19:58:59.883244 (+ 75us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
Metrics: {"spinlock_wait_cycles":62800128,"thread_start_us":1204750,"threads_started":1,"wal-append.queue_time_us":745094,"child_traces":[["op",{"prepare.queue_time_us":189211,"prepare.run_cpu_time_us":548,"prepare.run_wall_time_us":548}]]}
I20260926 19:59:00.923807 13403 ts_tablet_manager.cc:1950] T 89c8cbb097a644218e0ca033a6fdb4a2 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:59:00.925688 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 287) took 3811 ms. Trace:
I20260926 19:59:00.926159 13012 tablet_bootstrap.cc:492] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:59:00.925989 13403 rpcz_store.cc:276] 0926 19:58:57.114166 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.122658 (+ 8492us) service_pool.cc:224] Handling call
0926 19:59:00.252876 (+3130218us) ts_tablet_manager.cc:1151] Deleting tablet 89c8cbb097a644218e0ca033a6fdb4a2
0926 19:59:00.252930 (+ 54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:00.903938 (+651008us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.920161 (+ 16223us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.925632 (+ 5471us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":316351,"tablet-delete.queue_time_us":3128094}
I20260926 19:59:00.930061 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 31f10eb547ab4ab09e5b1e2a4f52bfcb (DEFAULT_TABLE table=default.Table_640b34d9f5ac410a82cf9c2ba23cd525 [id=7da8c8c422224467b70a086c631503a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.931129 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31f10eb547ab4ab09e5b1e2a4f52bfcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.904410 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet b32f8b0d480547a58ba5c6fed6483a74 with delete_type TABLET_DATA_DELETED (replaced by tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:00.565394 12730 rpcz_store.cc:276] 0926 19:58:59.239232 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.239397 (+ 165us) service_pool.cc:224] Handling call
0926 19:58:59.509081 (+269684us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.513589 (+ 4508us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:59.620279 (+106690us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.564478 (+944199us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:00.564975 (+ 497us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260926 19:59:00.940228 14027 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.321s user 0.005s sys 0.015s
I20260926 19:59:00.941411 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1dfed69d00a647d1814ee455d0d3b5e2" } new_table_name: "default.Table_636582145de0438fb4a11f4504144c31" modify_external_catalogs: true
I20260926 19:59:00.947366 12390 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1e0638a7221d47b19343a23c178452cc" } modify_external_catalogs: true
I20260926 19:59:00.947937 12390 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_1e0638a7221d47b19343a23c178452cc" } modify_external_catalogs: true
I20260926 19:59:00.941720 12723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0c465b503b54e569fbf529da3ac316c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.949266 14027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:33594 (request call id 298) took 2154 ms. Trace:
I20260926 19:59:00.949631 14027 rpcz_store.cc:276] 0926 19:58:58.795017 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:58.928494 (+133477us) service_pool.cc:224] Handling call
0926 19:59:00.949202 (+2020708us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:58:59.282520 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0926 19:58:59.282877 (+ 357us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0926 19:58:59.282923 (+ 46us) alter_schema_op.cc:66] Acquired schema lock
0926 19:58:59.282968 (+ 45us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0926 19:58:59.283088 (+ 120us) alter_schema_op.cc:128] START. Timestamp: 1790452739282995
0926 19:58:59.283109 (+ 21us) op_driver.cc:348] REPLICATION: starting
0926 19:58:59.283689 (+ 580us) log.cc:844] Serialized 173 byte log entry
0926 19:59:00.241277 (+957588us) op_driver.cc:464] REPLICATION: finished
0926 19:59:00.603337 (+362060us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0926 19:59:00.619170 (+ 15833us) log.cc:844] Serialized 5 byte log entry
0926 19:59:00.948318 (+329148us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.948973 (+ 655us) log.cc:844] Serialized 14 byte log entry
0926 19:59:00.949071 (+ 98us) alter_schema_op.cc:73] Released schema lock
0926 19:59:00.949100 (+ 29us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":361871,"prepare.queue_time_us":175141,"prepare.run_cpu_time_us":2078,"prepare.run_wall_time_us":85126,"raft.queue_time_us":586,"raft.run_cpu_time_us":1211,"raft.run_wall_time_us":19394,"replication_time_us":958044,"thread_start_us":849822,"threads_started":4,"wal-append.queue_time_us":322703,"wal-append.run_cpu_time_us":1347,"wal-append.run_wall_time_us":1037481}]]}
I20260926 19:59:00.938216 13962 consensus_queue.cc:1048] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260926 19:59:00.959013 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3972c0cc04894010ba6aac0586d60c5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.954842 12729 tablet_service.cc:1511] Processing CreateTablet for tablet 1a5b12bdb6f346d99fd27c57fcd1064f (DEFAULT_TABLE table=default.Table_2aae6d0b16cf43b8ade8e77279c797b8 [id=79430eea271b4a3381b5fc56600210f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:00.965062 12729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5b12bdb6f346d99fd27c57fcd1064f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.971737 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_50f95c20537b40ed86ee75b4619cd35d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:00.971817 12388 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_ba0bfeee8cac4c709fe66a3645e5b71e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:00.977939 12388 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba0bfeee8cac4c709fe66a3645e5b71e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:00.981448 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50f95c20537b40ed86ee75b4619cd35d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:00.991338 12390 catalog_manager.cc:6128] Not sending DeleteTablet requests; no consensus state for tablet 09d51613cd464e61af285ea6ff0f603b
I20260926 19:59:01.002957 13915 consensus_queue.cc:1048] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:01.003247 14050 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:59:01.003861 14050 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
W20260926 19:59:01.002568 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 72109.6us
W20260926 19:59:01.006098 12696 leader_election.cc:343] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:01.008488 12395 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_f136506f88074adcb77c8ab2bc88ce51" } new_table_name: "default.Table_5a07f3f43a144f119494bec46e9a844b" modify_external_catalogs: true
I20260926 19:59:01.012877 12695 leader_election.cc:304] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:59:01.021353 12730 tablet_service.cc:1511] Processing CreateTablet for tablet 57643cb928554e70a22c4b213411e0b2 (DEFAULT_TABLE table=default.Table_5284d05a66f641f1858d0b6dbf51ae4b [id=765c307ad69a4fb2a6befc0c55508351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.022856 12730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57643cb928554e70a22c4b213411e0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.011262 12731 tablet_service.cc:1511] Processing CreateTablet for tablet 8dbc482325c743e9964b5258b5cb620f (DEFAULT_TABLE table=default.Table_d39ede314be3449d96bbf00b8c9be90b [id=4a7273d6bbbd47568cc1c107b021161b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.026692 12731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dbc482325c743e9964b5258b5cb620f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.026365 14030 consensus_queue.cc:1048] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:01.032253 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7dc92330d400f93f427f3f79ad826" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:01.032871 12758 raft_consensus.cc:2468] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:59:01.035746 13962 raft_consensus.cc:2804] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:01.036234 13962 raft_consensus.cc:493] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:01.036728 13962 raft_consensus.cc:3060] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:59:01.038595 12695 consensus_peers.cc:597] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ad76fc959e24545bdfad19f988fce93. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:59:01.037443 13993 consensus_queue.cc:1048] T e27c5b06d33c43ab9a483e1fc9b18234 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:01.045164 13915 consensus_queue.cc:1048] T fcad0bbb2bb44338bd56f2547e70ab9c P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:59:01.047748 12248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 382 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:59:01.060182 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 382 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:59:01.062028 13739 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:59:01.065287 14050 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:59:01.065555 14052 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:01.074682 12695 leader_election.cc:304] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters: 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:59:01.075466 13993 raft_consensus.cc:2804] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:01.075824 13993 raft_consensus.cc:493] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:01.076169 13993 raft_consensus.cc:3060] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:01.076694 13012 tablet_bootstrap.cc:654] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9: Neither blocks nor log segments found. Creating new log.
W20260926 19:59:01.083441 12695 consensus_peers.cc:597] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f666e1a8d214172bf5b752e9ec374bd. This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:01.084595 12695 consensus_peers.cc:597] T 5930402db43740b4b291c6c238ecf52c P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5930402db43740b4b291c6c238ecf52c. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:59:01.098758 12807 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:01.101172 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb035b50e8f445ba492e67796085123" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:01.106009 12396 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3a61927510f1455ead6604257dcf158c" } modify_external_catalogs: true
I20260926 19:59:01.106961 12396 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_3a61927510f1455ead6604257dcf158c" } modify_external_catalogs: true
I20260926 19:59:01.107237 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 2cc8bbccd0bf403db22e3791b2fb854d (DEFAULT_TABLE table=default.Table_d6d28646357a4b26ba16bbb1a7eb01d9 [id=c4265098bdb34d0ab60545dd2293ba84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.110483 12568 tablet_service.cc:1558] Processing DeleteTablet for tablet dbd3a49039354c28b8faf1fd052210ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:01.127491 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cc8bbccd0bf403db22e3791b2fb854d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:00.927049 13403 tablet_replica.cc:333] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:59:01.138123 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a85852c08304e0fbbbcbb2c0d24b27e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:59 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:01.138172 13768 ts_manager.cc:194] Registered new tserver with Master: 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:59:01.139712 12741 tablet_service.cc:1558] Processing DeleteTablet for tablet 59f7dc92330d400f93f427f3f79ad826 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d153e60ae5d486cb91c2cc623b99985) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:01.140759 12741 tablet_service.cc:1511] Processing CreateTablet for tablet f6630cb35e584d488beb2b3f27355c19 (DEFAULT_TABLE table=default.Table_eb4069f7356a45b7bbbd58b430343d29 [id=16379e7ded6340149710638b5d04efc1]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:00.906674 13092 ts_tablet_manager.cc:1469] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Trace:
0926 19:58:45.410422 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:45.410551 (+ 129us) service_pool.cc:224] Handling call
0926 19:58:45.412381 (+ 1830us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:45.412462 (+ 81us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:45.421382 (+ 8920us) tablet_metadata.cc:665] Metadata flushed
0926 19:58:45.443763 (+ 22381us) tablet_replica.cc:206] Creating consensus instance
0926 19:58:45.444103 (+ 340us) inbound_call.cc:177] Queueing success response
0926 19:58:59.334343 (+13890240us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0926 19:59:00.308062 (+973719us) ts_tablet_manager.cc:1435] Starting tablet replica
0926 19:59:00.648211 (+340149us) tablet_replica.cc:261] Starting instrumentation
0926 19:59:00.648708 (+ 497us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":10032,"raft.queue_time_us":165352,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":44292,"tablet-open.queue_time_us":13890214}
I20260926 19:59:01.142225 12391 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_c5597da00ff04e24a0e5f257cd0d3353"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:01.143981 12391 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c5597da00ff04e24a0e5f257cd0d3353 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:01.147671 12395 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_272be26b54a546cf80b56c1c7b402235"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:00.875582 14017 raft_consensus.cc:2749] T 74fe405570f8474fb7a1f841a783a90a P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:00.878453 14001 raft_consensus.cc:2749] T 14ca451e5139456b9deb41a4fc635c87 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:01.154060 12594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb035b50e8f445ba492e67796085123" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:59:01.156071 14047 raft_consensus.cc:2749] T 54553d20d48148ae88fa4cc2ce021779 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:01.158250 13092 tablet_bootstrap.cc:492] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:59:00.886294 14015 raft_consensus.cc:515] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:59:01.161147 12894 tablet_service.cc:1558] Processing DeleteTablet for tablet 3488f0076b0f4ef19f7cf23a93cd20e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:01.161475 12890 tablet_service.cc:1511] Processing CreateTablet for tablet 2315702ccbe74c0197a834f1f59ad0df (DEFAULT_TABLE table=default.Table_b8021096ec5e4b29ada2934d6401a48d [id=0bad243ae1ea4cf2a5274e72ec863be5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:01.160617 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 284527us
W20260926 19:59:01.164803 12395 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_272be26b54a546cf80b56c1c7b402235 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:01.142530 12887 consensus_meta.cc:287] T 4d43853cbe2e438c8588460d7b9c1660 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent flushing consensus metadata: real 0.521s user 0.001s sys 0.006s
I20260926 19:59:01.165153 14043 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260926 19:59:01.154585 12594 raft_consensus.cc:2468] T edb035b50e8f445ba492e67796085123 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:59:01.169234 12895 tablet_service.cc:1511] Processing CreateTablet for tablet ab4b47bb2a7f4b2a855cbb299ef665de (DEFAULT_TABLE table=default.Table_b840208ddba74ea084e0e59efccb56f5 [id=635811e25a6d495b829d241ae4e35af7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.170379 12895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab4b47bb2a7f4b2a855cbb299ef665de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.171604 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ad76fc959e24545bdfad19f988fce93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:53 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:01.172648 14020 raft_consensus.cc:493] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:01.173975 13403 raft_consensus.cc:2243] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:01.173061 14020 raft_consensus.cc:515] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:01.176561 12559 tablet_service.cc:1511] Processing CreateTablet for tablet ddfc561dfbe9481bb327f3a2b7332e31 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.177139 14020 leader_election.cc:290] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:01.177495 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfc561dfbe9481bb327f3a2b7332e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.178948 13403 raft_consensus.cc:2272] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:01.195101 12388 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_98e4f6017318449bbfcf847f5d8b51ee" } modify_external_catalogs: true
I20260926 19:59:01.195643 12388 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_98e4f6017318449bbfcf847f5d8b51ee" } modify_external_catalogs: true
I20260926 19:59:01.199231 12390 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:45734:
table { table_name: "default.Table_b2cab18b22ac46d9bd64e47ac24851e0" } new_table_name: "default.Table_12e9e918e36241f6b13d30b9cdc2595f" modify_external_catalogs: true
I20260926 19:59:01.200883 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:01.203421 12555 tablet_service.cc:1511] Processing CreateTablet for tablet afe01ed8c0b04f42ac9cab125853c0f7 (DEFAULT_TABLE table=default.Table_f25b2e9ee0e441e9ab4fe05366a52e2d [id=df3fe609d944485294740a723335db0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.204530 12555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe01ed8c0b04f42ac9cab125853c0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.221975 14001 raft_consensus.cc:493] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:01.222695 14001 raft_consensus.cc:515] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:59:01.224601 14001 leader_election.cc:290] T fcad0bbb2bb44338bd56f2547e70ab9c P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:01.249387 13013 tablet_bootstrap.cc:654] T fe567f5d4c8c41ff9d681ca9ad3970e8 P f6e9d6439fd54b28925a68a6145d3dee: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:01.249467 12389 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_7d9b396d3f0a41fba07931d6ce8c143d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:01.251075 12389 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d9b396d3f0a41fba07931d6ce8c143d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:01.266660 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6630cb35e584d488beb2b3f27355c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.269454 12396 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet c4d86fa6b40b4fc49c4fa67b63792831
I20260926 19:59:01.288590 13092 tablet_bootstrap.cc:654] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:01.291065 12903 tablet_service.cc:1511] Processing CreateTablet for tablet 1a5b12bdb6f346d99fd27c57fcd1064f (DEFAULT_TABLE table=default.Table_2aae6d0b16cf43b8ade8e77279c797b8 [id=79430eea271b4a3381b5fc56600210f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.291843 12893 tablet_service.cc:1558] Processing DeleteTablet for tablet 59f7dc92330d400f93f427f3f79ad826 with delete_type TABLET_DATA_DELETED (replaced by tablet 4d153e60ae5d486cb91c2cc623b99985) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:01.293991 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 3972c0cc04894010ba6aac0586d60c5b (DEFAULT_TABLE table=default.Table_a9722bfab4444bfabadaa2118c2da3b9 [id=089b4f87dcdf4a5e9dc035b3bdbcd57f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:01.299412 12519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 157712us
I20260926 19:59:01.309393 12199 master-stress-test.cc:579] Killing test thread 1/10
I20260926 19:59:01.256021 14002 ts_tablet_manager.cc:1929] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260926 19:59:01.317890 12886 consensus_meta.cc:287] T 353fd263095d44ad8000a6a490565153 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent flushing consensus metadata: real 1.049s user 0.008s sys 0.002s
I20260926 19:59:01.325601 12887 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 320) took 1576 ms. Trace:
I20260926 19:59:01.285282 14015 leader_election.cc:290] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:01.325961 12887 rpcz_store.cc:276] 0926 19:58:59.749643 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.010419 (+260776us) service_pool.cc:224] Handling call
0926 19:59:00.198396 (+187977us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:00.198508 (+ 112us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:00.361968 (+163460us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.286226 (+924258us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.325546 (+ 39320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328}
W20260926 19:59:01.328176 12888 consensus_meta.cc:287] T 1c38a5aab5eb4404b783eaa5cc477c31 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent flushing consensus metadata: real 0.662s user 0.005s sys 0.001s
I20260926 19:59:01.292382 12890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2315702ccbe74c0197a834f1f59ad0df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.289301 12894 tablet_service.cc:1558] Processing DeleteTablet for tablet 5930402db43740b4b291c6c238ecf52c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:01.331641 13403 ts_tablet_manager.cc:1916] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:01.332362 12896 tablet_service.cc:1511] Processing CreateTablet for tablet 2cc8bbccd0bf403db22e3791b2fb854d (DEFAULT_TABLE table=default.Table_d6d28646357a4b26ba16bbb1a7eb01d9 [id=c4265098bdb34d0ab60545dd2293ba84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.332875 12887 tablet_service.cc:1511] Processing CreateTablet for tablet 57643cb928554e70a22c4b213411e0b2 (DEFAULT_TABLE table=default.Table_5284d05a66f641f1858d0b6dbf51ae4b [id=765c307ad69a4fb2a6befc0c55508351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.334702 12893 tablet_service.cc:1511] Processing CreateTablet for tablet 8dbc482325c743e9964b5258b5cb620f (DEFAULT_TABLE table=default.Table_d39ede314be3449d96bbf00b8c9be90b [id=4a7273d6bbbd47568cc1c107b021161b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.333904 12887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57643cb928554e70a22c4b213411e0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.336115 12886 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 321) took 1586 ms. Trace:
I20260926 19:59:01.336881 12903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5b12bdb6f346d99fd27c57fcd1064f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.338192 12894 tablet_service.cc:1511] Processing CreateTablet for tablet f6630cb35e584d488beb2b3f27355c19 (DEFAULT_TABLE table=default.Table_eb4069f7356a45b7bbbd58b430343d29 [id=16379e7ded6340149710638b5d04efc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.339347 12894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6630cb35e584d488beb2b3f27355c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.342267 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cc8bbccd0bf403db22e3791b2fb854d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.343748 12888 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 319) took 1594 ms. Trace:
I20260926 19:59:01.344127 12888 rpcz_store.cc:276] 0926 19:58:59.749313 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.010454 (+261141us) service_pool.cc:224] Handling call
0926 19:59:00.193101 (+182647us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:00.193209 (+ 108us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:00.597019 (+403810us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.343269 (+746250us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.343696 (+ 427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:01.336536 12886 rpcz_store.cc:276] 0926 19:58:59.749957 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.009092 (+259135us) service_pool.cc:224] Handling call
0926 19:59:00.188440 (+179348us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:00.188557 (+ 117us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:00.268785 (+ 80228us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.335645 (+1066860us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.336054 (+ 409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:01.365720 12394 master_service.cc:960] ReplaceTablet: received request to replace tablet a1e776237ca24a32a8e73d2f0628df3e from {username='slave'} at 127.0.0.1:45734
I20260926 19:59:01.370787 12892 tablet_service.cc:1511] Processing CreateTablet for tablet 658c75b17f13420f8207f87a18eb9359 (DEFAULT_TABLE table=default.Table_c8b681e9cf4b4c26a3ea0fb2f83c0cd6 [id=a56e39982bb24f56a1348ef03eaf0510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.372653 12902 tablet_service.cc:1511] Processing CreateTablet for tablet 34c89822201b44d1bee654ee75f8c4a7 (DEFAULT_TABLE table=default.Table_c341da116f104491a6da32f46b955ecd [id=66c76c7dc6134a3e876cc48d9e663f74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.374708 12891 tablet_service.cc:1511] Processing CreateTablet for tablet ddfc561dfbe9481bb327f3a2b7332e31 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.376834 12897 tablet_service.cc:1511] Processing CreateTablet for tablet afe01ed8c0b04f42ac9cab125853c0f7 (DEFAULT_TABLE table=default.Table_f25b2e9ee0e441e9ab4fe05366a52e2d [id=df3fe609d944485294740a723335db0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.379935 12901 tablet_service.cc:1511] Processing CreateTablet for tablet 96bc6f58797244bba597d6d724c1b267 (DEFAULT_TABLE table=default.Table_d148763f180a4d4aa0b91f6577410475 [id=d0c038726f5042b88de96a1b8405cfb4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:01.391687 12383 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 81918e7e00f84a0ab8e19963a9d20332 (table default.Table_8a7885640bb644a0abfa68ec902c48ed [id=f91be1fbbd294acf88688109480b4be2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 81918e7e00f84a0ab8e19963a9d20332 consensus state has no leader
W20260926 19:59:01.394263 12889 consensus_meta.cc:287] T 9ed22219df2b42ad86a6d7917ee0d990 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent flushing consensus metadata: real 1.138s user 0.007s sys 0.004s
I20260926 19:59:01.407214 12898 tablet_service.cc:1511] Processing CreateTablet for tablet b32ea3efccb748239be91e71cc60349c (DEFAULT_TABLE table=default.Table_9725558d90dd4ba587d2c6e8def7f565 [id=1ee6151a3eb141dcb6580291d73a0db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.400687 12886 tablet_service.cc:1511] Processing CreateTablet for tablet 708609d227e3431eb932c79289386afb (DEFAULT_TABLE table=default.Table_ffee871354884e4a882a0945a6a111c4 [id=36eb5997553f442395e21cb4693f1b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.397645 12888 tablet_service.cc:1558] Processing DeleteTablet for tablet e27c5b06d33c43ab9a483e1fc9b18234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:01.401644 12893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dbc482325c743e9964b5258b5cb620f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.430331 12898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b32ea3efccb748239be91e71cc60349c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.430841 12901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bc6f58797244bba597d6d724c1b267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.433154 12902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c89822201b44d1bee654ee75f8c4a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.437775 12891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfc561dfbe9481bb327f3a2b7332e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.441722 12897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe01ed8c0b04f42ac9cab125853c0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.442847 12736 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 316) took 2235 ms. Trace:
I20260926 19:59:01.444228 12737 tablet_service.cc:1558] Processing DeleteTablet for tablet e27c5b06d33c43ab9a483e1fc9b18234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:01.445254 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 708609d227e3431eb932c79289386afb (DEFAULT_TABLE table=default.Table_ffee871354884e4a882a0945a6a111c4 [id=36eb5997553f442395e21cb4693f1b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.446763 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 708609d227e3431eb932c79289386afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.447263 12892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 658c75b17f13420f8207f87a18eb9359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.461386 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 89c8cbb097a644218e0ca033a6fdb4a2 (table default.Table_27371e4eb0cd4fa48aaef20d2db04fcc [id=f5d5844ddbeb4542b5e7b02d22800604]) successfully deleted
W20260926 19:59:01.464968 12858 consensus_peers.cc:597] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b32f8b0d480547a58ba5c6fed6483a74. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:59:01.466586 12903 tablet_service.cc:1558] Processing DeleteTablet for tablet dbd3a49039354c28b8faf1fd052210ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:01.318188 14002 log.cc:1199] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/8c7d0435f3564cb38b6cbca713009c2e
I20260926 19:59:01.474546 14002 ts_tablet_manager.cc:1950] T 8c7d0435f3564cb38b6cbca713009c2e P f6e9d6439fd54b28925a68a6145d3dee: Deleting consensus metadata
I20260926 19:59:01.476279 14002 tablet_replica.cc:333] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: stopping tablet replica
I20260926 19:59:01.480777 14002 raft_consensus.cc:2243] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:01.481782 14002 raft_consensus.cc:2272] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:01.443203 12736 rpcz_store.cc:276] 0926 19:58:59.207270 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.207843 (+ 573us) service_pool.cc:224] Handling call
0926 19:58:59.508627 (+300784us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.508710 (+ 83us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:58:59.969305 (+460595us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:00.900422 (+931117us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.442781 (+542359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:01.483705 14002 ts_tablet_manager.cc:1916] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:01.490543 12595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "700d05149a934d5e8bec9cc41873ad3e" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6e9d6439fd54b28925a68a6145d3dee" is_pre_election: true
I20260926 19:58:59.259189 13787 rpcz_store.cc:276] 0926 19:58:56.156379 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:56.156924 (+ 545us) service_pool.cc:224] Handling call
0926 19:58:56.885294 (+728370us) raft_consensus.cc:1471] Updating replica for 374 ops
0926 19:58:57.200143 (+314849us) raft_consensus.cc:1534] Early marking committed up to index 8
0926 19:58:57.311806 (+111663us) raft_consensus.cc:1539] Triggering prepare for 374 ops
0926 19:58:57.312240 (+ 434us) write_op.cc:270] Start()
0926 19:58:57.312363 (+ 123us) write_op.cc:276] Timestamp: P: 1790452714911067 usec, L: 0
0926 19:58:57.338619 (+ 26256us) write_op.cc:270] Start()
0926 19:58:57.338916 (+ 297us) write_op.cc:276] Timestamp: P: 1790452714921897 usec, L: 0
0926 19:58:57.339560 (+ 644us) write_op.cc:270] Start()
0926 19:58:57.339649 (+ 89us) write_op.cc:276] Timestamp: P: 1790452714931680 usec, L: 0
0926 19:58:57.340155 (+ 506us) write_op.cc:270] Start()
0926 19:58:57.340264 (+ 109us) write_op.cc:276] Timestamp: P: 1790452714975307 usec, L: 0
0926 19:58:57.340800 (+ 536us) write_op.cc:270] Start()
0926 19:58:57.340900 (+ 100us) write_op.cc:276] Timestamp: P: 1790452714979346 usec, L: 0
0926 19:58:57.341354 (+ 454us) write_op.cc:270] Start()
0926 19:58:57.341443 (+ 89us) write_op.cc:276] Timestamp: P: 1790452714983613 usec, L: 0
0926 19:58:57.342010 (+ 567us) write_op.cc:270] Start()
0926 19:58:57.342145 (+ 135us) write_op.cc:276] Timestamp: P: 1790452714987584 usec, L: 0
0926 19:58:57.342612 (+ 467us) write_op.cc:270] Start()
0926 19:58:57.343565 (+ 953us) write_op.cc:276] Timestamp: P: 1790452715029530 usec, L: 0
0926 19:58:57.344278 (+ 713us) write_op.cc:270] Start()
0926 19:58:57.344377 (+ 99us) write_op.cc:276] Timestamp: P: 1790452715156517 usec, L: 0
0926 19:58:57.344940 (+ 563us) write_op.cc:270] Start()
0926 19:58:57.345037 (+ 97us) write_op.cc:276] Timestamp: P: 1790452715227475 usec, L: 0
0926 19:58:57.354882 (+ 9845us) write_op.cc:270] Start()
0926 19:58:57.354974 (+ 92us) write_op.cc:276] Timestamp: P: 1790452715269181 usec, L: 0
0926 19:58:57.355479 (+ 505us) write_op.cc:270] Start()
0926 19:58:57.355584 (+ 105us) write_op.cc:276] Timestamp: P: 1790452715274476 usec, L: 0
0926 19:58:57.356083 (+ 499us) write_op.cc:270] Start()
0926 19:58:57.356163 (+ 80us) write_op.cc:276] Timestamp: P: 1790452715284996 usec, L: 0
0926 19:58:57.356656 (+ 493us) write_op.cc:270] Start()
0926 19:58:57.356734 (+ 78us) write_op.cc:276] Timestamp: P: 1790452715347262 usec, L: 0
0926 19:58:57.357217 (+ 483us) write_op.cc:270] Start()
0926 19:58:57.357297 (+ 80us) write_op.cc:276] Timestamp: P: 1790452715411120 usec, L: 0
0926 19:58:57.357857 (+ 560us) write_op.cc:270] Start()
0926 19:58:57.357946 (+ 89us) write_op.cc:276] Timestamp: P: 1790452715487084 usec, L: 0
0926 19:58:57.358474 (+ 528us) write_op.cc:270] Start()
0926 19:58:57.358556 (+ 82us) write_op.cc:276] Timestamp: P: 1790452715510493 usec, L: 0
0926 19:58:57.359040 (+ 484us) write_op.cc:270] Start()
0926 19:58:57.359123 (+ 83us) write_op.cc:276] Timestamp: P: 1790452715569597 usec, L: 0
0926 19:58:57.365727 (+ 6604us) write_op.cc:270] Start()
0926 19:58:57.365830 (+ 103us) write_op.cc:276] Timestamp: P: 1790452715580157 usec, L: 0
0926 19:58:57.382408 (+ 16578us) write_op.cc:270] Start()
0926 19:58:57.382514 (+ 106us) write_op.cc:276] Timestamp: P: 1790452715638856 usec, L: 0
0926 19:58:57.383787 (+ 1273us) write_op.cc:270] Start()
0926 19:58:57.383891 (+ 104us) write_op.cc:276] Timestamp: P: 1790452715689205 usec, L: 0
0926 19:58:57.384459 (+ 568us) write_op.cc:270] Start()
0926 19:58:57.384550 (+ 91us) write_op.cc:276] Timestamp: P: 1790452715713178 usec, L: 0
0926 19:58:57.385094 (+ 544us) write_op.cc:270] Start()
0926 19:58:57.385172 (+ 78us) write_op.cc:276] Timestamp: P: 1790452715814370 usec, L: 0
0926 19:58:57.387236 (+ 2064us) write_op.cc:270] Start()
0926 19:58:57.387388 (+ 152us) write_op.cc:276] Timestamp: P: 1790452715850354 usec, L: 0
0926 19:58:57.388168 (+ 780us) write_op.cc:270] Start()
0926 19:58:57.388274 (+ 106us) write_op.cc:276] Timestamp: P: 1790452715893663 usec, L: 0
0926 19:58:57.388824 (+ 550us) write_op.cc:270] Start()
0926 19:58:57.388924 (+ 100us) write_op.cc:276] Timestamp: P: 1790452716027758 usec, L: 0
0926 19:58:57.396240 (+ 7316us) write_op.cc:270] Start()
0926 19:58:57.396359 (+ 119us) write_op.cc:276] Timestamp: P: 1790452716125319 usec, L: 0
0926 19:58:57.397031 (+ 672us) write_op.cc:270] Start()
0926 19:58:57.397129 (+ 98us) write_op.cc:276] Timestamp: P: 1790452716189821 usec, L: 0
0926 19:58:57.397727 (+ 598us) write_op.cc:270] Start()
0926 19:58:57.397827 (+ 100us) write_op.cc:276] Timestamp: P: 1790452716240642 usec, L: 0
0926 19:58:57.398380 (+ 553us) write_op.cc:270] Start()
0926 19:58:57.398477 (+ 97us) write_op.cc:276] Timestamp: P: 1790452716325841 usec, L: 0
0926 19:58:57.398984 (+ 507us) write_op.cc:270] Start()
0926 19:58:57.399092 (+ 108us) write_op.cc:276] Timestamp: P: 1790452716346091 usec, L: 0
0926 19:58:57.399647 (+ 555us) write_op.cc:270] Start()
0926 19:58:57.399742 (+ 95us) write_op.cc:276] Timestamp: P: 1790452716349155 usec, L: 0
0926 19:58:57.400326 (+ 584us) write_op.cc:270] Start()
0926 19:58:57.400412 (+ 86us) write_op.cc:276] Timestamp: P: 1790452716357137 usec, L: 0
0926 19:58:57.400993 (+ 581us) write_op.cc:270] Start()
0926 19:58:57.401083 (+ 90us) write_op.cc:276] Timestamp: P: 1790452716359325 usec, L: 0
0926 19:58:57.402013 (+ 930us) write_op.cc:270] Start()
0926 19:58:57.402101 (+ 88us) write_op.cc:276] Timestamp: P: 1790452716375108 usec, L: 0
0926 19:58:57.402580 (+ 479us) write_op.cc:270] Start()
0926 19:58:57.402652 (+ 72us) write_op.cc:276] Timestamp: P: 1790452716405346 usec, L: 0
0926 19:58:57.403105 (+ 453us) write_op.cc:270] Start()
0926 19:58:57.403181 (+ 76us) write_op.cc:276] Timestamp: P: 1790452716425188 usec, L: 0
0926 19:58:57.403644 (+ 463us) write_op.cc:270] Start()
0926 19:58:57.403735 (+ 91us) write_op.cc:276] Timestamp: P: 1790452716458981 usec, L: 0
0926 19:58:57.404213 (+ 478us) write_op.cc:270] Start()
0926 19:58:57.404289 (+ 76us) write_op.cc:276] Timestamp: P: 1790452716505478 usec, L: 0
0926 19:58:57.404766 (+ 477us) write_op.cc:270] Start()
0926 19:58:57.404847 (+ 81us) write_op.cc:276] Timestamp: P: 1790452716538728 usec, L: 0
0926 19:58:57.405342 (+ 495us) write_op.cc:270] Start()
0926 19:58:57.405425 (+ 83us) write_op.cc:276] Timestamp: P: 1790452716582210 usec, L: 0
0926 19:58:57.405919 (+ 494us) write_op.cc:270] Start()
0926 19:58:57.406009 (+ 90us) write_op.cc:276] Timestamp: P: 1790452716635219 usec, L: 0
0926 19:58:57.406664 (+ 655us) write_op.cc:270] Start()
0926 19:58:57.406737 (+ 73us) write_op.cc:276] Timestamp: P: 1790452716647305 usec, L: 0
0926 19:58:57.407302 (+ 565us) write_op.cc:270] Start()
0926 19:58:57.407380 (+ 78us) write_op.cc:276] Timestamp: P: 1790452716681320 usec, L: 0
0926 19:58:57.407930 (+ 550us) write_op.cc:270] Start()
0926 19:58:57.408098 (+ 168us) write_op.cc:276] Timestamp: P: 1790452716699399 usec, L: 0
0926 19:58:57.409423 (+ 1325us) write_op.cc:270] Start()
0926 19:58:57.409532 (+ 109us) write_op.cc:276] Timestamp: P: 1790452716835575 usec, L: 0
0926 19:58:57.426458 (+ 16926us) write_op.cc:270] Start()
0926 19:58:57.426552 (+ 94us) write_op.cc:276] Timestamp: P: 1790452716843048 usec, L: 0
0926 19:58:57.427043 (+ 491us) write_op.cc:270] Start()
0926 19:58:57.427244 (+ 201us) write_op.cc:276] Timestamp: P: 1790452717056504 usec, L: 0
0926 19:58:57.427887 (+ 643us) write_op.cc:270] Start()
0926 19:58:57.427971 (+ 84us) write_op.cc:276] Timestamp: P: 1790452717058821 usec, L: 0
0926 19:58:57.428690 (+ 719us) write_op.cc:270] Start()
0926 19:58:57.428765 (+ 75us) write_op.cc:276] Timestamp: P: 1790452717061753 usec, L: 0
0926 19:58:57.429726 (+ 961us) write_op.cc:270] Start()
0926 19:58:57.429833 (+ 107us) write_op.cc:276] Timestamp: P: 1790452717063437 usec, L: 0
0926 19:58:57.430934 (+ 1101us) write_op.cc:270] Start()
0926 19:58:57.431023 (+ 89us) write_op.cc:276] Timestamp: P: 1790452717309714 usec, L: 0
0926 19:58:57.431509 (+ 486us) write_op.cc:270] Start()
0926 19:58:57.431591 (+ 82us) write_op.cc:276] Timestamp: P: 1790452717334774 usec, L: 0
0926 19:58:57.432082 (+ 491us) write_op.cc:270] Start()
0926 19:58:57.432169 (+ 87us) write_op.cc:276] Timestamp: P: 1790452717369555 usec, L: 0
0926 19:58:57.432630 (+ 461us) write_op.cc:270] Start()
0926 19:58:57.432716 (+ 86us) write_op.cc:276] Timestamp: P: 1790452717371326 usec, L: 0
0926 19:58:57.433175 (+ 459us) write_op.cc:270] Start()
0926 19:58:57.433274 (+ 99us) write_op.cc:276] Timestamp: P: 1790452717386305 usec, L: 0
0926 19:58:57.433755 (+ 481us) write_op.cc:270] Start()
0926 19:58:57.433827 (+ 72us) write_op.cc:276] Timestamp: P: 1790452717416870 usec, L: 0
0926 19:58:57.434262 (+ 435us) write_op.cc:270] Start()
0926 19:58:57.434339 (+ 77us) write_op.cc:276] Timestamp: P: 1790452717420735 usec, L: 0
0926 19:58:57.434843 (+ 504us) write_op.cc:270] Start()
0926 19:58:57.434915 (+ 72us) write_op.cc:276] Timestamp: P: 1790452717623212 usec, L: 0
0926 19:58:57.435339 (+ 424us) write_op.cc:270] Start()
0926 19:58:57.435489 (+ 150us) write_op.cc:276] Timestamp: P: 1790452717625263 usec, L: 0
0926 19:58:57.435930 (+ 441us) write_op.cc:270] Start()
0926 19:58:57.436014 (+ 84us) write_op.cc:276] Timestamp: P: 1790452717635329 usec, L: 0
0926 19:58:57.436511 (+ 497us) write_op.cc:270] Start()
0926 19:58:57.436593 (+ 82us) write_op.cc:276] Timestamp: P: 1790452717713940 usec, L: 0
0926 19:58:57.437086 (+ 493us) write_op.cc:270] Start()
0926 19:58:57.437211 (+ 125us) write_op.cc:276] Timestamp: P: 1790452717717379 usec, L: 0
0926 19:58:57.437823 (+ 612us) write_op.cc:270] Start()
0926 19:58:57.437895 (+ 72us) write_op.cc:276] Timestamp: P: 1790452717726929 usec, L: 0
0926 19:58:57.438520 (+ 625us) write_op.cc:270] Start()
0926 19:58:57.438597 (+ 77us) write_op.cc:276] Timestamp: P: 1790452717874732 usec, L: 0
0926 19:58:57.439151 (+ 554us) write_op.cc:270] Start()
0926 19:58:57.439237 (+ 86us) write_op.cc:276] Timestamp: P: 1790452717879066 usec, L: 0
0926 19:58:57.439678 (+ 441us) write_op.cc:270] Start()
0926 19:58:57.439754 (+ 76us) write_op.cc:276] Timestamp: P: 1790452717899710 usec, L: 0
0926 19:58:57.440222 (+ 468us) write_op.cc:270] Start()
0926 19:58:57.440301 (+ 79us) write_op.cc:276] Timestamp: P: 1790452717904383 usec, L: 0
0926 19:58:57.440832 (+ 531us) write_op.cc:270] Start()
0926 19:58:57.440903 (+ 71us) write_op.cc:276] Timestamp: P: 1790452717955160 usec, L: 0
0926 19:58:57.441347 (+ 444us) write_op.cc:270] Start()
0926 19:58:57.441432 (+ 85us) write_op.cc:276] Timestamp: P: 1790452717959494 usec, L: 0
0926 19:58:57.441907 (+ 475us) write_op.cc:270] Start()
0926 19:58:57.441980 (+ 73us) write_op.cc:276] Timestamp: P: 1790452718011983 usec, L: 0
0926 19:58:57.442541 (+ 561us) write_op.cc:270] Start()
0926 19:58:57.442621 (+ 80us) write_op.cc:276] Timestamp: P: 1790452718094653 usec, L: 0
0926 19:58:57.443110 (+ 489us) write_op.cc:270] Start()
0926 19:58:57.443188 (+ 78us) write_op.cc:276] Timestamp: P: 1790452718131483 usec, L: 0
0926 19:58:57.443964 (+ 776us) write_op.cc:270] Start()
0926 19:58:57.444067 (+ 103us) write_op.cc:276] Timestamp: P: 1790452718134160 usec, L: 0
0926 19:58:57.444582 (+ 515us) write_op.cc:270] Start()
0926 19:58:57.444661 (+ 79us) write_op.cc:276] Timestamp: P: 1790452718179693 usec, L: 0
0926 19:58:57.445142 (+ 481us) write_op.cc:270] Start()
0926 19:58:57.445223 (+ 81us) write_op.cc:276] Timestamp: P: 1790452718204771 usec, L: 0
0926 19:58:57.445726 (+ 503us) write_op.cc:270] Start()
0926 19:58:57.445808 (+ 82us) write_op.cc:276] Timestamp: P: 1790452718217412 usec, L: 0
0926 19:58:57.446311 (+ 503us) write_op.cc:270] Start()
0926 19:58:57.446394 (+ 83us) write_op.cc:276] Timestamp: P: 1790452718224931 usec, L: 0
0926 19:58:57.446826 (+ 432us) write_op.cc:270] Start()
0926 19:58:57.446903 (+ 77us) write_op.cc:276] Timestamp: P: 1790452718287540 usec, L: 0
0926 19:58:57.447375 (+ 472us) write_op.cc:270] Start()
0926 19:58:57.447446 (+ 71us) write_op.cc:276] Timestamp: P: 1790452718319648 usec, L: 0
0926 19:58:57.447857 (+ 411us) write_op.cc:270] Start()
0926 19:58:57.447924 (+ 67us) write_op.cc:276] Timestamp: P: 1790452718331470 usec, L: 0
0926 19:58:57.448437 (+ 513us) write_op.cc:270] Start()
0926 19:58:57.448511 (+ 74us) write_op.cc:276] Timestamp: P: 1790452718338353 usec, L: 0
0926 19:58:57.448955 (+ 444us) write_op.cc:270] Start()
0926 19:58:57.449052 (+ 97us) write_op.cc:276] Timestamp: P: 1790452718344013 usec, L: 0
0926 19:58:57.449542 (+ 490us) write_op.cc:270] Start()
0926 19:58:57.449625 (+ 83us) write_op.cc:276] Timestamp: P: 1790452718491471 usec, L: 0
0926 19:58:57.450088 (+ 463us) write_op.cc:270] Start()
0926 19:58:57.450170 (+ 82us) write_op.cc:276] Timestamp: P: 1790452718501587 usec, L: 0
0926 19:58:57.450616 (+ 446us) write_op.cc:270] Start()
0926 19:58:57.450700 (+ 84us) write_op.cc:276] Timestamp: P: 1790452718525061 usec, L: 0
0926 19:58:57.458246 (+ 7546us) write_op.cc:270] Start()
0926 19:58:57.458334 (+ 88us) write_op.cc:276] Timestamp: P: 1790452718598967 usec, L: 0
0926 19:58:57.458865 (+ 531us) write_op.cc:270] Start()
0926 19:58:57.458946 (+ 81us) write_op.cc:276] Timestamp: P: 1790452718623873 usec, L: 0
0926 19:58:57.459416 (+ 470us) write_op.cc:270] Start()
0926 19:58:57.459494 (+ 78us) write_op.cc:276] Timestamp: P: 1790452718638138 usec, L: 0
0926 19:58:57.460854 (+ 1360us) write_op.cc:270] Start()
0926 19:58:57.460937 (+ 83us) write_op.cc:276] Timestamp: P: 1790452718640155 usec, L: 0
0926 19:58:57.461405 (+ 468us) write_op.cc:270] Start()
0926 19:58:57.461497 (+ 92us) write_op.cc:276] Timestamp: P: 1790452718740209 usec, L: 0
0926 19:58:57.462005 (+ 508us) write_op.cc:270] Start()
0926 19:58:57.462082 (+ 77us) write_op.cc:276] Timestamp: P: 1790452718818385 usec, L: 0
0926 19:58:57.462537 (+ 455us) write_op.cc:270] Start()
0926 19:58:57.462617 (+ 80us) write_op.cc:276] Timestamp: P: 1790452718821207 usec, L: 0
0926 19:58:57.475007 (+ 12390us) write_op.cc:270] Start()
0926 19:58:57.475119 (+ 112us) write_op.cc:276] Timestamp: P: 1790452718963354 usec, L: 0
0926 19:58:57.476140 (+ 1021us) write_op.cc:270] Start()
0926 19:58:57.476230 (+ 90us) write_op.cc:276] Timestamp: P: 1790452718965561 usec, L: 0
0926 19:58:57.476787 (+ 557us) write_op.cc:270] Start()
0926 19:58:57.476862 (+ 75us) write_op.cc:276] Timestamp: P: 1790452719026455 usec, L: 0
0926 19:58:57.477320 (+ 458us) write_op.cc:270] Start()
0926 19:58:57.477394 (+ 74us) write_op.cc:276] Timestamp: P: 1790452719028551 usec, L: 0
0926 19:58:57.477900 (+ 506us) write_op.cc:270] Start()
0926 19:58:57.478114 (+ 214us) write_op.cc:276] Timestamp: P: 1790452719058672 usec, L: 0
0926 19:58:57.478688 (+ 574us) write_op.cc:270] Start()
0926 19:58:57.478772 (+ 84us) write_op.cc:276] Timestamp: P: 1790452719140477 usec, L: 0
0926 19:58:57.479251 (+ 479us) write_op.cc:270] Start()
0926 19:58:57.479323 (+ 72us) write_op.cc:276] Timestamp: P: 1790452719159280 usec, L: 0
0926 19:58:57.479782 (+ 459us) write_op.cc:270] Start()
0926 19:58:57.479858 (+ 76us) write_op.cc:276] Timestamp: P: 1790452719171698 usec, L: 0
0926 19:58:57.480378 (+ 520us) write_op.cc:270] Start()
0926 19:58:57.480472 (+ 94us) write_op.cc:276] Timestamp: P: 1790452719201256 usec, L: 0
0926 19:58:57.481434 (+ 962us) write_op.cc:270] Start()
0926 19:58:57.481528 (+ 94us) write_op.cc:276] Timestamp: P: 1790452719219005 usec, L: 0
0926 19:58:57.482057 (+ 529us) write_op.cc:270] Start()
0926 19:58:57.482133 (+ 76us) write_op.cc:276] Timestamp: P: 1790452719257271 usec, L: 0
0926 19:58:57.482567 (+ 434us) write_op.cc:270] Start()
0926 19:58:57.482642 (+ 75us) write_op.cc:276] Timestamp: P: 1790452719390890 usec, L: 0
0926 19:58:57.483099 (+ 457us) write_op.cc:270] Start()
0926 19:58:57.483162 (+ 63us) write_op.cc:276] Timestamp: P: 1790452719395424 usec, L: 0
0926 19:58:57.483566 (+ 404us) write_op.cc:270] Start()
0926 19:58:57.483639 (+ 73us) write_op.cc:276] Timestamp: P: 1790452719410233 usec, L: 0
0926 19:58:57.484065 (+ 426us) write_op.cc:270] Start()
0926 19:58:57.489662 (+ 5597us) write_op.cc:276] Timestamp: P: 1790452719458104 usec, L: 0
0926 19:58:57.490157 (+ 495us) write_op.cc:270] Start()
0926 19:58:57.490228 (+ 71us) write_op.cc:276] Timestamp: P: 1790452719621471 usec, L: 0
0926 19:58:57.490627 (+ 399us) write_op.cc:270] Start()
0926 19:58:57.490802 (+ 175us) write_op.cc:276] Timestamp: P: 1790452719707015 usec, L: 0
0926 19:58:57.491238 (+ 436us) write_op.cc:270] Start()
0926 19:58:57.491308 (+ 70us) write_op.cc:276] Timestamp: P: 1790452719708953 usec, L: 0
0926 19:58:57.491718 (+ 410us) write_op.cc:270] Start()
0926 19:58:57.491786 (+ 68us) write_op.cc:276] Timestamp: P: 1790452719712316 usec, L: 0
0926 19:58:57.492208 (+ 422us) write_op.cc:270] Start()
0926 19:58:57.492288 (+ 80us) write_op.cc:276] Timestamp: P: 1790452719848857 usec, L: 0
0926 19:58:57.492732 (+ 444us) write_op.cc:270] Start()
0926 19:58:57.492885 (+ 153us) write_op.cc:276] Timestamp: P: 1790452719857682 usec, L: 0
0926 19:58:57.493309 (+ 424us) write_op.cc:270] Start()
0926 19:58:57.493370 (+ 61us) write_op.cc:276] Timestamp: P: 1790452719875879 usec, L: 0
0926 19:58:57.493943 (+ 573us) write_op.cc:270] Start()
0926 19:58:57.494023 (+ 80us) write_op.cc:276] Timestamp: P: 1790452719975863 usec, L: 0
0926 19:58:57.494833 (+ 810us) write_op.cc:270] Start()
0926 19:58:57.494902 (+ 69us) write_op.cc:276] Timestamp: P: 1790452720058578 usec, L: 0
0926 19:58:57.501378 (+ 6476us) write_op.cc:270] Start()
0926 19:58:57.501455 (+ 77us) write_op.cc:276] Timestamp: P: 1790452720086313 usec, L: 0
0926 19:58:57.501941 (+ 486us) write_op.cc:270] Start()
0926 19:58:57.502019 (+ 78us) write_op.cc:276] Timestamp: P: 1790452720119283 usec, L: 0
0926 19:58:57.502466 (+ 447us) write_op.cc:270] Start()
0926 19:58:57.502534 (+ 68us) write_op.cc:276] Timestamp: P: 1790452720143873 usec, L: 0
0926 19:58:57.502945 (+ 411us) write_op.cc:270] Start()
0926 19:58:57.503026 (+ 81us) write_op.cc:276] Timestamp: P: 1790452720177897 usec, L: 0
0926 19:58:57.503449 (+ 423us) write_op.cc:270] Start()
0926 19:58:57.503521 (+ 72us) write_op.cc:276] Timestamp: P: 1790452720317576 usec, L: 0
0926 19:58:57.503916 (+ 395us) write_op.cc:270] Start()
0926 19:58:57.503990 (+ 74us) write_op.cc:276] Timestamp: P: 1790452720372474 usec, L: 0
0926 19:58:57.504422 (+ 432us) write_op.cc:270] Start()
0926 19:58:57.504484 (+ 62us) write_op.cc:276] Timestamp: P: 1790452720376305 usec, L: 0
0926 19:58:57.504910 (+ 426us) write_op.cc:270] Start()
0926 19:58:57.505018 (+ 108us) write_op.cc:276] Timestamp: P: 1790452720411091 usec, L: 0
0926 19:58:57.505597 (+ 579us) write_op.cc:270] Start()
0926 19:58:57.505662 (+ 65us) write_op.cc:276] Timestamp: P: 1790452720414958 usec, L: 0
0926 19:58:57.506105 (+ 443us) write_op.cc:270] Start()
0926 19:58:57.506176 (+ 71us) write_op.cc:276] Timestamp: P: 1790452720429045 usec, L: 0
0926 19:58:57.506782 (+ 606us) write_op.cc:270] Start()
0926 19:58:57.506870 (+ 88us) write_op.cc:276] Timestamp: P: 1790452720446893 usec, L: 0
0926 19:58:57.507826 (+ 956us) write_op.cc:270] Start()
0926 19:58:57.509111 (+ 1285us) write_op.cc:276] Timestamp: P: 1790452720499450 usec, L: 0
0926 19:58:57.509568 (+ 457us) write_op.cc:270] Start()
0926 19:58:57.509640 (+ 72us) write_op.cc:276] Timestamp: P: 1790452720501285 usec, L: 0
0926 19:58:57.510058 (+ 418us) write_op.cc:270] Start()
0926 19:58:57.510217 (+ 159us) write_op.cc:276] Timestamp: P: 1790452720537534 usec, L: 0
0926 19:58:57.510702 (+ 485us) write_op.cc:270] Start()
0926 19:58:57.510763 (+ 61us) write_op.cc:276] Timestamp: P: 1790452720748616 usec, L: 0
0926 19:58:57.511222 (+ 459us) write_op.cc:270] Start()
0926 19:58:57.511325 (+ 103us) write_op.cc:276] Timestamp: P: 1790452720806956 usec, L: 0
0926 19:58:57.511815 (+ 490us) write_op.cc:270] Start()
0926 19:58:57.511903 (+ 88us) write_op.cc:276] Timestamp: P: 1790452720812205 usec, L: 0
0926 19:58:57.512423 (+ 520us) write_op.cc:270] Start()
0926 19:58:57.512519 (+ 96us) write_op.cc:276] Timestamp: P: 1790452720815592 usec, L: 0
0926 19:58:57.512923 (+ 404us) write_op.cc:270] Start()
0926 19:58:57.512981 (+ 58us) write_op.cc:276] Timestamp: P: 1790452720817767 usec, L: 0
0926 19:58:57.513345 (+ 364us) write_op.cc:270] Start()
0926 19:58:57.513404 (+ 59us) write_op.cc:276] Timestamp: P: 1790452720848552 usec, L: 0
0926 19:58:57.513841 (+ 437us) write_op.cc:270] Start()
0926 19:58:57.513906 (+ 65us) write_op.cc:276] Timestamp: P: 1790452720911735 usec, L: 0
0926 19:58:57.514284 (+ 378us) write_op.cc:270] Start()
0926 19:58:57.514349 (+ 65us) write_op.cc:276] Timestamp: P: 1790452720959422 usec, L: 0
0926 19:58:57.514716 (+ 367us) write_op.cc:270] Start()
0926 19:58:57.514777 (+ 61us) write_op.cc:276] Timestamp: P: 1790452721021422 usec, L: 0
0926 19:58:57.515150 (+ 373us) write_op.cc:270] Start()
0926 19:58:57.515210 (+ 60us) write_op.cc:276] Timestamp: P: 1790452721076097 usec, L: 0
0926 19:58:57.515562 (+ 352us) write_op.cc:270] Start()
0926 19:58:57.515621 (+ 59us) write_op.cc:276] Timestamp: P: 1790452721081211 usec, L: 0
0926 19:58:57.515986 (+ 365us) write_op.cc:270] Start()
0926 19:58:57.516065 (+ 79us) write_op.cc:276] Timestamp: P: 1790452721094335 usec, L: 0
0926 19:58:57.516425 (+ 360us) write_op.cc:270] Start()
0926 19:58:57.516513 (+ 88us) write_op.cc:276] Timestamp: P: 1790452721107306 usec, L: 0
0926 19:58:57.516888 (+ 375us) write_op.cc:270] Start()
0926 19:58:57.516949 (+ 61us) write_op.cc:276] Timestamp: P: 1790452721109044 usec, L: 0
0926 19:58:57.517317 (+ 368us) write_op.cc:270] Start()
0926 19:58:57.517384 (+ 67us) write_op.cc:276] Timestamp: P: 1790452721199590 usec, L: 0
0926 19:58:57.517830 (+ 446us) write_op.cc:270] Start()
0926 19:58:57.536592 (+ 18762us) write_op.cc:276] Timestamp: P: 1790452721224512 usec, L: 0
0926 19:58:57.537161 (+ 569us) write_op.cc:270] Start()
0926 19:58:57.537246 (+ 85us) write_op.cc:276] Timestamp: P: 1790452721259249 usec, L: 0
0926 19:58:57.538040 (+ 794us) write_op.cc:270] Start()
0926 19:58:57.538112 (+ 72us) write_op.cc:276] Timestamp: P: 1790452721302864 usec, L: 0
0926 19:58:57.538573 (+ 461us) write_op.cc:270] Start()
0926 19:58:57.538640 (+ 67us) write_op.cc:276] Timestamp: P: 1790452721412081 usec, L: 0
0926 19:58:57.539090 (+ 450us) write_op.cc:270] Start()
0926 19:58:57.539308 (+ 218us) write_op.cc:276] Timestamp: P: 1790452721416264 usec, L: 0
0926 19:58:57.539726 (+ 418us) write_op.cc:270] Start()
0926 19:58:57.539798 (+ 72us) write_op.cc:276] Timestamp: P: 1790452721419177 usec, L: 0
0926 19:58:57.540217 (+ 419us) write_op.cc:270] Start()
0926 19:58:57.540285 (+ 68us) write_op.cc:276] Timestamp: P: 1790452721422370 usec, L: 0
0926 19:58:57.540938 (+ 653us) write_op.cc:270] Start()
0926 19:58:57.541018 (+ 80us) write_op.cc:276] Timestamp: P: 1790452721437152 usec, L: 0
0926 19:58:57.541427 (+ 409us) write_op.cc:270] Start()
0926 19:58:57.541533 (+ 106us) write_op.cc:276] Timestamp: P: 1790452721480919 usec, L: 0
0926 19:58:57.541934 (+ 401us) write_op.cc:270] Start()
0926 19:58:57.542013 (+ 79us) write_op.cc:276] Timestamp: P: 1790452721582643 usec, L: 0
0926 19:58:57.542433 (+ 420us) write_op.cc:270] Start()
0926 19:58:57.542500 (+ 67us) write_op.cc:276] Timestamp: P: 1790452721648241 usec, L: 0
0926 19:58:57.543029 (+ 529us) write_op.cc:270] Start()
0926 19:58:57.543102 (+ 73us) write_op.cc:276] Timestamp: P: 1790452721806671 usec, L: 0
0926 19:58:57.543514 (+ 412us) write_op.cc:270] Start()
0926 19:58:57.543594 (+ 80us) write_op.cc:276] Timestamp: P: 1790452721843451 usec, L: 0
0926 19:58:57.544019 (+ 425us) write_op.cc:270] Start()
0926 19:58:57.544094 (+ 75us) write_op.cc:276] Timestamp: P: 1790452721887741 usec, L: 0
0926 19:58:57.545776 (+ 1682us) write_op.cc:270] Start()
0926 19:58:57.545860 (+ 84us) write_op.cc:276] Timestamp: P: 1790452722006795 usec, L: 0
0926 19:58:57.546284 (+ 424us) write_op.cc:270] Start()
0926 19:58:57.546348 (+ 64us) write_op.cc:276] Timestamp: P: 1790452722017370 usec, L: 0
0926 19:58:57.546776 (+ 428us) write_op.cc:270] Start()
0926 19:58:57.546854 (+ 78us) write_op.cc:276] Timestamp: P: 1790452722019093 usec, L: 0
0926 19:58:57.553731 (+ 6877us) write_op.cc:270] Start()
0926 19:58:57.553807 (+ 76us) write_op.cc:276] Timestamp: P: 1790452722074667 usec, L: 0
0926 19:58:57.554292 (+ 485us) write_op.cc:270] Start()
0926 19:58:57.554355 (+ 63us) write_op.cc:276] Timestamp: P: 1790452722076501 usec, L: 0
0926 19:58:57.554775 (+ 420us) write_op.cc:270] Start()
0926 19:58:57.554856 (+ 81us) write_op.cc:276] Timestamp: P: 1790452722144043 usec, L: 0
0926 19:58:57.555400 (+ 544us) write_op.cc:270] Start()
0926 19:58:57.555484 (+ 84us) write_op.cc:276] Timestamp: P: 1790452722145894 usec, L: 0
0926 19:58:57.555917 (+ 433us) write_op.cc:270] Start()
0926 19:58:57.556025 (+ 108us) write_op.cc:276] Timestamp: P: 1790452722303523 usec, L: 0
0926 19:58:57.556531 (+ 506us) write_op.cc:270] Start()
0926 19:58:57.556620 (+ 89us) write_op.cc:276] Timestamp: P: 1790452722311882 usec, L: 0
0926 19:58:57.557148 (+ 528us) write_op.cc:270] Start()
0926 19:58:57.557238 (+ 90us) write_op.cc:276] Timestamp: P: 1790452722328145 usec, L: 0
0926 19:58:57.569093 (+ 11855us) write_op.cc:270] Start()
0926 19:58:57.569178 (+ 85us) write_op.cc:276] Timestamp: P: 1790452722335585 usec, L: 0
0926 19:58:57.571175 (+ 1997us) write_op.cc:270] Start()
0926 19:58:57.571252 (+ 77us) write_op.cc:276] Timestamp: P: 1790452722381692 usec, L: 0
0926 19:58:57.571711 (+ 459us) write_op.cc:270] Start()
0926 19:58:57.571777 (+ 66us) write_op.cc:276] Timestamp: P: 1790452722434095 usec, L: 0
0926 19:58:57.573907 (+ 2130us) write_op.cc:270] Start()
0926 19:58:57.573972 (+ 65us) write_op.cc:276] Timestamp: P: 1790452722546157 usec, L: 0
0926 19:58:57.574403 (+ 431us) write_op.cc:270] Start()
0926 19:58:57.574473 (+ 70us) write_op.cc:276] Timestamp: P: 1790452722684014 usec, L: 0
0926 19:58:57.575796 (+ 1323us) write_op.cc:270] Start()
0926 19:58:57.575861 (+ 65us) write_op.cc:276] Timestamp: P: 1790452722702261 usec, L: 0
0926 19:58:57.576272 (+ 411us) write_op.cc:270] Start()
0926 19:58:57.576340 (+ 68us) write_op.cc:276] Timestamp: P: 1790452722717486 usec, L: 0
0926 19:58:57.576743 (+ 403us) write_op.cc:270] Start()
0926 19:58:57.576806 (+ 63us) write_op.cc:276] Timestamp: P: 1790452722775313 usec, L: 0
0926 19:58:57.577208 (+ 402us) write_op.cc:270] Start()
0926 19:58:57.577271 (+ 63us) write_op.cc:276] Timestamp: P: 1790452722807553 usec, L: 0
0926 19:58:57.579777 (+ 2506us) write_op.cc:270] Start()
0926 19:58:57.579848 (+ 71us) write_op.cc:276] Timestamp: P: 1790452722833830 usec, L: 0
0926 19:58:57.580278 (+ 430us) write_op.cc:270] Start()
0926 19:58:57.580343 (+ 65us) write_op.cc:276] Timestamp: P: 1790452722861552 usec, L: 0
0926 19:58:57.580722 (+ 379us) write_op.cc:270] Start()
0926 19:58:57.580787 (+ 65us) write_op.cc:276] Timestamp: P: 1790452723049577 usec, L: 0
0926 19:58:57.582910 (+ 2123us) write_op.cc:270] Start()
0926 19:58:57.583031 (+ 121us) write_op.cc:276] Timestamp: P: 1790452723082587 usec, L: 0
0926 19:58:57.583536 (+ 505us) write_op.cc:270] Start()
0926 19:58:57.583618 (+ 82us) write_op.cc:276] Timestamp: P: 1790452723156588 usec, L: 0
0926 19:58:57.584078 (+ 460us) write_op.cc:270] Start()
0926 19:58:57.584160 (+ 82us) write_op.cc:276] Timestamp: P: 1790452723216274 usec, L: 0
0926 19:58:57.584631 (+ 471us) write_op.cc:270] Start()
0926 19:58:57.584721 (+ 90us) write_op.cc:276] Timestamp: P: 1790452723253076 usec, L: 0
0926 19:58:57.585192 (+ 471us) write_op.cc:270] Start()
0926 19:58:57.585271 (+ 79us) write_op.cc:276] Timestamp: P: 1790452723337963 usec, L: 0
0926 19:58:57.588507 (+ 3236us) write_op.cc:270] Start()
0926 19:58:57.588584 (+ 77us) write_op.cc:276] Timestamp: P: 1790452723364891 usec, L: 0
0926 19:58:57.589033 (+ 449us) write_op.cc:270] Start()
0926 19:58:57.589098 (+ 65us) write_op.cc:276] Timestamp: P: 1790452723367955 usec, L: 0
0926 19:58:57.589531 (+ 433us) write_op.cc:270] Start()
0926 19:58:57.589600 (+ 69us) write_op.cc:276] Timestamp: P: 1790452723386249 usec, L: 0
0926 19:58:57.590034 (+ 434us) write_op.cc:270] Start()
0926 19:58:57.590107 (+ 73us) write_op.cc:276] Timestamp: P: 1790452723392740 usec, L: 0
0926 19:58:57.590590 (+ 483us) write_op.cc:270] Start()
0926 19:58:57.590676 (+ 86us) write_op.cc:276] Timestamp: P: 1790452723401168 usec, L: 0
0926 19:58:57.591171 (+ 495us) write_op.cc:270] Start()
0926 19:58:57.591251 (+ 80us) write_op.cc:276] Timestamp: P: 1790452723408050 usec, L: 0
0926 19:58:57.591726 (+ 475us) write_op.cc:270] Start()
0926 19:58:57.591805 (+ 79us) write_op.cc:276] Timestamp: P: 1790452723429737 usec, L: 0
0926 19:58:57.592273 (+ 468us) write_op.cc:270] Start()
0926 19:58:57.592355 (+ 82us) write_op.cc:276] Timestamp: P: 1790452723465130 usec, L: 0
0926 19:58:57.592798 (+ 443us) write_op.cc:270] Start()
0926 19:58:57.597638 (+ 4840us) write_op.cc:276] Timestamp: P: 1790452723525258 usec, L: 0
0926 19:58:57.598127 (+ 489us) write_op.cc:270] Start()
0926 19:58:57.598195 (+ 68us) write_op.cc:276] Timestamp: P: 17904527
I20260926 19:59:01.499905 13768 ts_manager.cc:194] Registered new tserver with Master: 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:59:01.520190 12858 leader_election.cc:343] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 59f7dc92330d400f93f427f3f79ad826
W20260926 19:59:01.525780 12858 leader_election.cc:343] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Illegal state: Tablet not RUNNING: STOPPED
W20260926 19:59:01.526337 12858 leader_election.cc:343] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: cd86f01d82d041f4852f83f2547e697f
I20260926 19:59:01.519963 12886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 708609d227e3431eb932c79289386afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.547865 12734 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 318) took 2262 ms. Trace:
I20260926 19:59:01.554188 14025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } } }
I20260926 19:59:01.554989 14025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:01.561726 12388 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet 986a28107bd54199bf2cda96978150c3
I20260926 19:59:01.548241 12734 rpcz_store.cc:276] 0926 19:58:59.285079 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.285209 (+ 130us) service_pool.cc:224] Handling call
0926 19:58:59.506431 (+221222us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.506516 (+ 85us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:00.606956 (+1100440us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.445650 (+838694us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.547798 (+102148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:01.569790 12734 tablet_service.cc:1558] Processing DeleteTablet for tablet dbd3a49039354c28b8faf1fd052210ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:00 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:01.460127 12895 tablet_service.cc:1558] Processing DeleteTablet for tablet c4d86fa6b40b4fc49c4fa67b63792831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:01 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:01.574251 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3972c0cc04894010ba6aac0586d60c5b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:01.576982 12738 consensus_meta.cc:287] T 1c38a5aab5eb4404b783eaa5cc477c31 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.516s user 0.001s sys 0.004s
I20260926 19:59:01.582646 12885 tablet_service.cc:1558] Processing DeleteTablet for tablet 986a28107bd54199bf2cda96978150c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:01 UTC) from {username='slave'} at 127.0.0.1:33594
W20260926 19:59:01.583370 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 128474us
I20260926 19:59:01.584016 12396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45734:
name: "default.Table_380d5450416f4fca9ceebc9625ca9c9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:01.585176 12888 tablet_service.cc:1511] Processing CreateTablet for tablet c1ec11f9e2f546d0b0ce2b1bc3cc49a5 (DEFAULT_TABLE table=default.Table_eca0d16a6176401c9021c2153e65b550 [id=bd4ee4e9347b4e5bad7c88dc23e151d6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:01.586272 12396 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_380d5450416f4fca9ceebc9625ca9c9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:01.586530 12888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1ec11f9e2f546d0b0ce2b1bc3cc49a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.591408 12735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 334) took 2167 ms. Trace:
W20260926 19:59:01.594038 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 456801us
I20260926 19:59:01.544488 12889 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 318) took 1795 ms. Trace:
I20260926 19:59:01.614267 12889 rpcz_store.cc:276] 0926 19:58:59.748938 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.010376 (+261438us) service_pool.cc:224] Handling call
0926 19:59:00.159939 (+149563us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:00.160201 (+ 262us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:00.255813 (+ 95612us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.444617 (+1188804us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.539782 (+ 95165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:01.625108 13092 tablet_bootstrap.cc:492] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
W20260926 19:59:01.642625 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 479339us
W20260926 19:59:01.647210 12521 leader_election.cc:343] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:01.647642 12521 leader_election.cc:304] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6e9d6439fd54b28925a68a6145d3dee; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5
I20260926 19:59:01.639925 12545 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37786 (request call id 50) took 1333 ms. Trace:
I20260926 19:59:01.672427 13092 ts_tablet_manager.cc:1403] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.514s user 0.018s sys 0.005s
W20260926 19:59:01.659359 12857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:37461, user_credentials={real_user=slave}} blocked reactor thread for 53907.1us
I20260926 19:59:01.673275 12857 leader_election.cc:304] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:01.676810 14017 raft_consensus.cc:2749] T 3066a65ce44b4387a6f96124d9f476fb P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:01.679531 12381 catalog_manager.cc:5063] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): tablet 8c7d0435f3564cb38b6cbca713009c2e (table default.Table_f05b7ded126448f2acba6af9bc981450 [id=ade55e2b356b4c9db57a1dd04b307ee1]) successfully deleted
W20260926 19:59:01.680685 12381 catalog_manager.cc:5030] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): delete failed for tablet 56fd4c88f9304f19883c3e5da8f937b5 because the tablet was not found. No further retry: Not found: Tablet not found: 56fd4c88f9304f19883c3e5da8f937b5
W20260926 19:59:01.687749 12857 consensus_peers.cc:597] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260926 19:59:01.692564 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 658c75b17f13420f8207f87a18eb9359 (DEFAULT_TABLE table=default.Table_c8b681e9cf4b4c26a3ea0fb2f83c0cd6 [id=a56e39982bb24f56a1348ef03eaf0510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.702102 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 658c75b17f13420f8207f87a18eb9359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.591739 12735 rpcz_store.cc:276] 0926 19:58:59.423830 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.494744 (+ 70914us) service_pool.cc:224] Handling call
0926 19:58:59.878638 (+383894us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.878735 (+ 97us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.065186 (+1186451us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.484881 (+419695us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.591347 (+106466us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:01.664124 13403 ts_tablet_manager.cc:1929] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:01.710712 13403 log.cc:1199] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/cd86f01d82d041f4852f83f2547e697f
I20260926 19:59:01.711445 13403 ts_tablet_manager.cc:1950] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:59:01.711443 12735 tablet_service.cc:1511] Processing CreateTablet for tablet ddfc561dfbe9481bb327f3a2b7332e31 (DEFAULT_TABLE table=default.Table_cd8d867057984c05bb2d20afcbeebbe2 [id=6a42d7a8facb45bf824a15ea1f9feede]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.713418 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddfc561dfbe9481bb327f3a2b7332e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.648527 12545 rpcz_store.cc:276] 0926 19:59:00.275387 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.275508 (+ 121us) service_pool.cc:224] Handling call
0926 19:59:00.714288 (+438780us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54000e38d4. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a258406b8 00007f9a268b198d 00007f9a25001104 00007f9a25001092 00007f9a25000fc1 00007f9a25000f3c 00007f9a250001f2 00007f9a2351509c 00007f9a235144d6 00007f9a23513e7f
0926 19:59:01.483248 (+768960us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b54000e3dd4. stack: 00007f9a1e6b2323 00007f9a1e6b2220 00007f9a1e256574 0000000000516964 00000000005168f9 00007f9a257afaa1 00007f9a258406b8 00007f9a268b198d 00007f9a25001104 00007f9a25001092 00007f9a25000fc1 00007f9a25000f3c 00007f9a250001f2 00007f9a2351509c 00007f9a235144d6 00007f9a23513e7f
0926 19:59:01.608913 (+125665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68461440}
I20260926 19:59:01.698925 14020 raft_consensus.cc:2804] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:01.698822 14018 consensus_queue.cc:1048] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.064s
I20260926 19:59:01.768702 12914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:01.746016 14020 raft_consensus.cc:493] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:01.780169 14020 raft_consensus.cc:3060] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:01.827977 13013 tablet_bootstrap.cc:492] T fe567f5d4c8c41ff9d681ca9ad3970e8 P f6e9d6439fd54b28925a68a6145d3dee: No bootstrap required, opened a new log
I20260926 19:59:01.828399 13013 ts_tablet_manager.cc:1403] T fe567f5d4c8c41ff9d681ca9ad3970e8 P f6e9d6439fd54b28925a68a6145d3dee: Time spent bootstrapping tablet: real 1.106s user 0.022s sys 0.002s
I20260926 19:59:01.831151 13092 raft_consensus.cc:359] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:01.832001 13092 raft_consensus.cc:385] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:01.832304 13092 raft_consensus.cc:740] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:59:01.851842 12567 tablet_service.cc:1511] Processing CreateTablet for tablet b32ea3efccb748239be91e71cc60349c (DEFAULT_TABLE table=default.Table_9725558d90dd4ba587d2c6e8def7f565 [id=1ee6151a3eb141dcb6580291d73a0db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.852867 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b32ea3efccb748239be91e71cc60349c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.869355 13092 consensus_queue.cc:260] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:01.872133 13092 ts_tablet_manager.cc:1434] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.199s user 0.005s sys 0.000s
I20260926 19:59:01.875168 14017 raft_consensus.cc:493] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:01.875602 14017 raft_consensus.cc:515] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:01.878120 13092 tablet_bootstrap.cc:492] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:59:01.877434 14017 leader_election.cc:290] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
I20260926 19:59:01.879738 14043 tablet.cc:1751] T 59d1955b12dd47899b8a226fe0e980ae P f6e9d6439fd54b28925a68a6145d3dee: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:59:01.719303 12741 tablet_service.cc:1511] Processing CreateTablet for tablet afe01ed8c0b04f42ac9cab125853c0f7 (DEFAULT_TABLE table=default.Table_f25b2e9ee0e441e9ab4fe05366a52e2d [id=df3fe609d944485294740a723335db0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.889984 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe01ed8c0b04f42ac9cab125853c0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.894006 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 289) took 4779 ms. Trace:
I20260926 19:59:01.893924 13013 raft_consensus.cc:359] T fe567f5d4c8c41ff9d681ca9ad3970e8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:01.894316 13403 rpcz_store.cc:276] 0926 19:58:57.114680 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.125015 (+ 10335us) service_pool.cc:224] Handling call
0926 19:59:00.926913 (+3801898us) ts_tablet_manager.cc:1151] Deleting tablet cd86f01d82d041f4852f83f2547e697f
0926 19:59:00.926953 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:01.178737 (+251784us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b54001225d4. stack: 00007ffa5c912323 00007ffa5c912220 00007ffa5c4b6574 0000000000516964 00000000005168f9 00007ffa63a0faa1 00007ffa63aaeb22 00007ffa63eef55c 00007ffa64b58c3a 00007ffa64b89960 00007ffa64b6fe6d 00007ffa64b6fe19 00007ffa64b6fda9 00007ffa64b6fd71 00007ffa64b6ef9d 00000000005bd4e4
0926 19:59:01.639447 (+460710us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.663976 (+ 24529us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.893944 (+229968us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7637,"spinlock_wait_cycles":68808960,"tablet-delete.queue_time_us":3801344}
I20260926 19:59:01.895938 13403 tablet_replica.cc:333] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:59:01.896714 13403 raft_consensus.cc:2243] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:59:01.897437 13403 pending_rounds.cc:70] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Trying to abort 1 pending ops.
I20260926 19:59:01.904194 14020 raft_consensus.cc:515] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:59:01.908195 12858 leader_election.cc:343] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Not found: Tablet not found: 59f7dc92330d400f93f427f3f79ad826
W20260926 19:59:01.913381 12923 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s user 0.009s sys 0.016s
W20260926 19:59:01.913740 12923 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s user 0.009s sys 0.016s
W20260926 19:59:01.914867 12924 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s user 0.007s sys 0.015s
W20260926 19:59:01.915184 12924 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s user 0.007s sys 0.015s
I20260926 19:59:01.957168 12387 master_service.cc:960] ReplaceTablet: received request to replace tablet fd98f43d7000438da50cc5285a4395ae from {username='slave'} at 127.0.0.1:45734
I20260926 19:59:01.897697 13403 pending_rounds.cc:77] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694410948231168 op_type: NO_OP noop_request { }
I20260926 19:59:01.961360 13403 raft_consensus.cc:2889] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:59:01.961784 13403 raft_consensus.cc:2272] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:01.968240 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7645307e0999482bbe3329c8d61cc567" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:01.970073 13569 ts_tablet_manager.cc:1929] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:01.970533 13569 log.cc:1199] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/22f268b4a1f7423f917731c6b62aab3e
I20260926 19:59:01.971487 14020 leader_election.cc:290] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
W20260926 19:59:01.548401 12757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s user 0.017s sys 0.008s
I20260926 19:59:01.975966 12738 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 333) took 2552 ms. Trace:
I20260926 19:59:01.978667 12737 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 358) took 1027 ms. Trace:
I20260926 19:59:01.982581 14089 tablet.cc:1751] T 59d1955b12dd47899b8a226fe0e980ae P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260926 19:59:01.988533 13092 tablet_bootstrap.cc:654] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:01.990885 13403 ts_tablet_manager.cc:1916] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:01.991616 12740 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 342) took 1579 ms. Trace:
I20260926 19:59:01.993113 12733 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 353) took 1076 ms. Trace:
I20260926 19:59:01.994192 12733 rpcz_store.cc:276] 0926 19:59:00.916289 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.916520 (+ 231us) service_pool.cc:224] Handling call
0926 19:59:00.918150 (+ 1630us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:00.918226 (+ 76us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.468388 (+550162us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.975369 (+506981us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.993041 (+ 17672us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:01.996533 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 96bc6f58797244bba597d6d724c1b267 (DEFAULT_TABLE table=default.Table_d148763f180a4d4aa0b91f6577410475 [id=d0c038726f5042b88de96a1b8405cfb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:01.999195 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bc6f58797244bba597d6d724c1b267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.979149 12737 rpcz_store.cc:276] 0926 19:59:00.950810 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.444649 (+493839us) service_pool.cc:224] Handling call
0926 19:59:01.446387 (+ 1738us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.446459 (+ 72us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.519874 (+ 73415us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.909340 (+389466us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.978612 (+ 69272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:01.976406 12738 rpcz_store.cc:276] 0926 19:58:59.423587 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.461953 (+ 38366us) service_pool.cc:224] Handling call
0926 19:58:59.696486 (+234533us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de7ce61 00007f2c9de5a0f3 00007f2c9de08640 00007f2c9c392bf4 00007f2c9c392b82 00007f2c9c392ab1
0926 19:58:59.696585 (+ 99us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.696656 (+ 71us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.061016 (+1364360us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.975464 (+914448us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.975900 (+ 436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11767040}
I20260926 19:59:02.012858 12737 tablet_service.cc:1511] Processing CreateTablet for tablet b32ea3efccb748239be91e71cc60349c (DEFAULT_TABLE table=default.Table_9725558d90dd4ba587d2c6e8def7f565 [id=1ee6151a3eb141dcb6580291d73a0db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.014293 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b32ea3efccb748239be91e71cc60349c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.020390 12734 tablet_service.cc:1511] Processing CreateTablet for tablet 34c89822201b44d1bee654ee75f8c4a7 (DEFAULT_TABLE table=default.Table_c341da116f104491a6da32f46b955ecd [id=66c76c7dc6134a3e876cc48d9e663f74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.021461 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c89822201b44d1bee654ee75f8c4a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.991930 12740 rpcz_store.cc:276] 0926 19:59:00.412436 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.413915 (+ 1479us) service_pool.cc:224] Handling call
0926 19:59:00.415041 (+ 1126us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:00.415111 (+ 70us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.469545 (+1054434us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.703779 (+234234us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:01.991559 (+287780us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.050067 12199 master-stress-test.cc:581] Killed test thread 1/10
I20260926 19:59:02.050390 12199 master-stress-test.cc:579] Killing test thread 2/10
I20260926 19:59:01.911796 12562 tablet_service.cc:1511] Processing CreateTablet for tablet c1ec11f9e2f546d0b0ce2b1bc3cc49a5 (DEFAULT_TABLE table=default.Table_eca0d16a6176401c9021c2153e65b550 [id=bd4ee4e9347b4e5bad7c88dc23e151d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.074286 12562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1ec11f9e2f546d0b0ce2b1bc3cc49a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:01.912909 12557 tablet_service.cc:1558] Processing DeleteTablet for tablet c4d86fa6b40b4fc49c4fa67b63792831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:01 UTC) from {username='slave'} at 127.0.0.1:37792
I20260926 19:59:02.097613 13989 raft_consensus.cc:493] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ae87963ed5e4d92a896bf4a73af4da9)
I20260926 19:59:02.098296 13989 raft_consensus.cc:515] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:02.119484 13989 leader_election.cc:290] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
W20260926 19:59:02.121567 12695 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s user 0.005s sys 0.029s
W20260926 19:59:02.121937 12695 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s user 0.005s sys 0.029s
I20260926 19:59:01.702598 13993 raft_consensus.cc:515] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:02.131655 12923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.11.233.194:55801 (request call id 301) took 1008 ms. Trace:
I20260926 19:59:02.127144 13915 raft_consensus.cc:493] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:02.133793 12757 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.11.233.193:37639 (request call id 239) took 1636 ms. Trace:
I20260926 19:59:02.142655 12757 rpcz_store.cc:276] 0926 19:59:00.497775 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.505128 (+ 7353us) service_pool.cc:224] Handling call
0926 19:59:00.505624 (+ 496us) raft_consensus.cc:1471] Updating replica for 1 ops
0926 19:59:00.505953 (+ 329us) raft_consensus.cc:1534] Early marking committed up to index 0
0926 19:59:00.505983 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0926 19:59:00.506310 (+ 327us) alter_schema_op.cc:128] START. Timestamp: 1790452740424746
0926 19:59:00.995825 (+489515us) log.cc:844] Serialized 173 byte log entry
0926 19:59:02.118849 (+1123024us) raft_consensus.cc:1645] Marking committed up to 1
0926 19:59:02.119533 (+ 684us) raft_consensus.cc:1697] Filling consensus response to leader.
0926 19:59:02.127751 (+ 8218us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0926 19:59:02.129112 (+ 1361us) raft_consensus.cc:1685] finished
0926 19:59:02.129703 (+ 591us) raft_consensus.cc:1691] UpdateReplicas() finished
0926 19:59:02.133139 (+ 3436us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:59:00.973228 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0926 19:59:00.979285 (+ 6057us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0926 19:59:00.979318 (+ 33us) alter_schema_op.cc:66] Acquired schema lock
0926 19:59:00.980525 (+ 1207us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
Metrics: {"thread_start_us":981983,"threads_started":1,"wal-append.queue_time_us":500819,"child_traces":[["op",{"prepare.queue_time_us":448538,"prepare.run_cpu_time_us":867,"prepare.run_wall_time_us":19698,"thread_start_us":402033,"threads_started":1}]]}
I20260926 19:59:02.146696 13993 leader_election.cc:290] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:02.145635 12923 rpcz_store.cc:276] 0926 19:59:01.123560 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.123660 (+ 100us) service_pool.cc:224] Handling call
0926 19:59:01.123915 (+ 255us) raft_consensus.cc:1471] Updating replica for 1 ops
0926 19:59:01.914069 (+790154us) raft_consensus.cc:1534] Early marking committed up to index 0
0926 19:59:01.914097 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0926 19:59:01.914485 (+ 388us) log.cc:844] Serialized 26 byte log entry
0926 19:59:02.093637 (+179152us) raft_consensus.cc:1645] Marking committed up to 0
0926 19:59:02.093672 (+ 35us) raft_consensus.cc:1697] Filling consensus response to leader.
0926 19:59:02.131399 (+ 37727us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0926 19:59:02.131442 (+ 43us) raft_consensus.cc:1685] finished
0926 19:59:02.131452 (+ 10us) raft_consensus.cc:1691] UpdateReplicas() finished
0926 19:59:02.131577 (+ 125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13487,"raft.queue_time_us":36944,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":955134,"threads_started":2,"wal-append.queue_time_us":173744}
I20260926 19:59:02.147912 13915 raft_consensus.cc:515] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:59:02.150813 12695 leader_election.cc:304] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee; no voters:
I20260926 19:59:02.151945 14030 raft_consensus.cc:2804] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:02.152261 14030 raft_consensus.cc:493] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:02.152572 14030 raft_consensus.cc:3060] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:59:02.156739 14009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.152s user 0.014s sys 0.021s
W20260926 19:59:02.157050 14009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.715s user 0.014s sys 0.021s
I20260926 19:59:02.140684 13403 ts_tablet_manager.cc:1929] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:02.166553 13403 log.cc:1199] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/acdb4bd8cd58444cb1d6d4b5610de20d
I20260926 19:59:02.167438 13403 ts_tablet_manager.cc:1950] T acdb4bd8cd58444cb1d6d4b5610de20d P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
W20260926 19:59:02.258555 12696 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s user 0.005s sys 0.005s
W20260926 19:59:02.259006 12696 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s user 0.005s sys 0.005s
W20260926 19:59:02.259279 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 675293us
I20260926 19:59:01.913942 12556 tablet_service.cc:1558] Processing DeleteTablet for tablet 986a28107bd54199bf2cda96978150c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:01 UTC) from {username='slave'} at 127.0.0.1:37792
W20260926 19:59:02.303761 12728 consensus_meta.cc:287] T 83a78d487f0147fb95e8b946539ab3a6 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 1.254s user 0.003s sys 0.022s
I20260926 19:59:02.305058 12728 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 329) took 2882 ms. Trace:
I20260926 19:59:02.312119 12732 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 352) took 1405 ms. Trace:
I20260926 19:59:02.312542 12732 rpcz_store.cc:276] 0926 19:59:00.906933 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.106496 (+199563us) service_pool.cc:224] Handling call
0926 19:59:01.127038 (+ 20542us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.127124 (+ 86us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.001964 (+874840us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.311694 (+309730us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.312064 (+ 370us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.321039 12758 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.11.233.195:39815 (request call id 144) took 1249 ms. Trace:
I20260926 19:59:02.322126 12758 rpcz_store.cc:276] 0926 19:59:01.071466 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.442267 (+370801us) service_pool.cc:224] Handling call
0926 19:59:02.018496 (+576229us) raft_consensus.cc:1471] Updating replica for 1 ops
0926 19:59:02.018841 (+ 345us) raft_consensus.cc:1534] Early marking committed up to index 0
0926 19:59:02.018877 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0926 19:59:02.019336 (+ 459us) log.cc:844] Serialized 26 byte log entry
0926 19:59:02.320565 (+301229us) raft_consensus.cc:1645] Marking committed up to 0
0926 19:59:02.320623 (+ 58us) raft_consensus.cc:1697] Filling consensus response to leader.
0926 19:59:02.320734 (+ 111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0926 19:59:02.320786 (+ 52us) raft_consensus.cc:1685] finished
0926 19:59:02.320808 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0926 19:59:02.320963 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":146,"thread_start_us":300804,"threads_started":1,"wal-append.queue_time_us":298884}
I20260926 19:59:01.934978 14002 ts_tablet_manager.cc:1929] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:01.936041 12559 tablet_service.cc:1511] Processing CreateTablet for tablet 96bc6f58797244bba597d6d724c1b267 (DEFAULT_TABLE table=default.Table_d148763f180a4d4aa0b91f6577410475 [id=d0c038726f5042b88de96a1b8405cfb4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260926 19:59:01.978370 12521 leader_election.cc:343] T 7645307e0999482bbe3329c8d61cc567 P f6e9d6439fd54b28925a68a6145d3dee [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:59:02.331111 12521 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 352767us
I20260926 19:59:02.074872 13013 raft_consensus.cc:385] T fe567f5d4c8c41ff9d681ca9ad3970e8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:02.332293 13013 raft_consensus.cc:740] T fe567f5d4c8c41ff9d681ca9ad3970e8 P f6e9d6439fd54b28925a68a6145d3dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e9d6439fd54b28925a68a6145d3dee, State: Initialized, Role: FOLLOWER
W20260926 19:59:02.333139 12739 consensus_meta.cc:287] T 2315702ccbe74c0197a834f1f59ad0df P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.786s user 0.013s sys 0.017s
W20260926 19:59:02.335129 13962 consensus_meta.cc:287] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 1.298s user 0.004s sys 0.014s
I20260926 19:59:02.335464 13962 raft_consensus.cc:515] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:02.305408 12728 rpcz_store.cc:276] 0926 19:58:59.422550 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.427875 (+ 5325us) service_pool.cc:224] Handling call
0926 19:58:59.700980 (+273105us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de7ce61 00007f2c9de5a0f3 00007f2c9de08640 00007f2c9c392bf4 00007f2c9c392b82 00007f2c9c392ab1
0926 19:58:59.701092 (+ 112us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.701352 (+ 260us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.039823 (+1338471us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.304466 (+1264643us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.305000 (+ 534us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75997440}
W20260926 19:59:02.339546 12722 consensus_meta.cc:287] T 036b9a2a0d65468ab8ff19ba06ec72a0 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 1.331s user 0.015s sys 0.002s
I20260926 19:59:02.341382 12394 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet a1e776237ca24a32a8e73d2f0628df3e
I20260926 19:59:02.341773 13915 leader_election.cc:290] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:02.343039 12394 catalog_manager.cc:6934] ReplaceTablet: tablet a1e776237ca24a32a8e73d2f0628df3e deleted and replaced by tablet 50d6cbf3191d4043b79f421a26d5f5ec
I20260926 19:59:02.354584 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet a1e776237ca24a32a8e73d2f0628df3e with delete_type TABLET_DATA_DELETED (replaced by tablet 50d6cbf3191d4043b79f421a26d5f5ec) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:02.357726 13569 ts_tablet_manager.cc:1950] T 22f268b4a1f7423f917731c6b62aab3e P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting consensus metadata
I20260926 19:59:02.358642 12739 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 344) took 1859 ms. Trace:
I20260926 19:59:02.359054 12739 rpcz_store.cc:276] 0926 19:59:00.498995 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.499120 (+ 125us) service_pool.cc:224] Handling call
0926 19:59:00.500403 (+ 1283us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:00.500499 (+ 96us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.509052 (+1008553us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.333571 (+824519us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.358583 (+ 25012us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.362020 13092 tablet_bootstrap.cc:492] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:59:02.362821 13092 ts_tablet_manager.cc:1403] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.485s user 0.025s sys 0.022s
I20260926 19:59:02.364166 12724 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 326) took 2942 ms. Trace:
I20260926 19:59:02.371227 12393 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_ac3b4dbe4f0241d09ea35f62599b84a5 [id=dd5c0a6b728948278a5dd48f8ec0bf6f] alter complete (version 4)
I20260926 19:59:02.375665 14020 raft_consensus.cc:493] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ae87963ed5e4d92a896bf4a73af4da9)
I20260926 19:59:02.376343 14020 raft_consensus.cc:515] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
W20260926 19:59:02.378286 12393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.11.233.195:37269 (request call id 72) took 1045 ms (client timeout 1000 ms). Trace:
I20260926 19:59:02.379360 14020 leader_election.cc:290] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:02.381070 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet 700d05149a934d5e8bec9cc41873ad3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:45 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:02.389906 14020 raft_consensus.cc:493] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ae87963ed5e4d92a896bf4a73af4da9)
I20260926 19:59:02.390507 14020 raft_consensus.cc:515] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:02.393278 12396 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:45734 (request call id 440) took 1094 ms. Trace:
I20260926 19:59:02.393574 14020 leader_election.cc:290] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:02.375041 12559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96bc6f58797244bba597d6d724c1b267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.393633 12396 rpcz_store.cc:276] 0926 19:59:01.298845 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.308111 (+ 9266us) service_pool.cc:224] Handling call
0926 19:59:01.583917 (+275806us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad8566 00007fb36a683034 00007fb36a682fc2 00007fb36a682ef1 00007fb36a682e6c 00007fb36a682122 00007fb36892d09c 00007fb36892c4d6
0926 19:59:01.611158 (+ 27241us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:59:01.612635 (+ 1477us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:59:02.392704 (+780069us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:59:02.392978 (+ 274us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:59:02.393222 (+ 244us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:59:02.369145 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:59:02.369561 (+ 416us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:59:02.369584 (+ 23us) write_op.cc:435] Acquired schema lock
0926 19:59:02.369596 (+ 12us) tablet.cc:662] Decoding operations
0926 19:59:02.369858 (+ 262us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:59:02.369897 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0926 19:59:02.369919 (+ 22us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:59:02.370101 (+ 182us) tablet.cc:701] Row locks acquired
0926 19:59:02.370115 (+ 14us) write_op.cc:260] PREPARE: finished
0926 19:59:02.370213 (+ 98us) write_op.cc:270] Start()
0926 19:59:02.370286 (+ 73us) write_op.cc:276] Timestamp: P: 1790452742370156 usec, L: 0
0926 19:59:02.370301 (+ 15us) op_driver.cc:348] REPLICATION: starting
0926 19:59:02.378941 (+ 8640us) log.cc:844] Serialized 433 byte log entry
0926 19:59:02.379504 (+ 563us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36ac5a5e8 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11 00007fb36b0edd3d 00000000005bd4e4 00000000005bd319
0926 19:59:02.389794 (+ 10290us) op_driver.cc:464] REPLICATION: finished
0926 19:59:02.391068 (+ 1274us) write_op.cc:301] APPLY: starting
0926 19:59:02.391125 (+ 57us) tablet.cc:1394] starting BulkCheckPresence
0926 19:59:02.391229 (+ 104us) tablet.cc:1397] finished BulkCheckPresence
0926 19:59:02.391257 (+ 28us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:59:02.391658 (+ 401us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:59:02.391784 (+ 126us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:59:02.391826 (+ 42us) write_op.cc:312] APPLY: finished
0926 19:59:02.392016 (+ 190us) log.cc:844] Serialized 29 byte log entry
0926 19:59:02.392139 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:59:02.392199 (+ 60us) write_op.cc:454] Released schema lock
0926 19:59:02.392228 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0926 19:59:02.392460 (+ 232us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:59:02.392482 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":632221952,"child_traces":[["op",{"apply.queue_time_us":1105,"apply.run_cpu_time_us":1356,"apply.run_wall_time_us":1356,"num_ops":2,"prepare.queue_time_us":751314,"prepare.run_cpu_time_us":2473,"prepare.run_wall_time_us":14294,"raft.queue_time_us":2266,"raft.run_cpu_time_us":628,"raft.run_wall_time_us":627,"replication_time_us":19389,"spinlock_wait_cycles":18825984}]]}
W20260926 19:59:02.396660 12568 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.625s user 0.009s sys 0.007s
W20260926 19:59:02.409461 12381 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet e1b4aff22b9e4932beff6c2b90a9ab1e (table default.Table_fdb8ab2956a44ea4af05d57738e9abcd [id=1e63508a8a7a4445b2d36f273cce75a0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e1b4aff22b9e4932beff6c2b90a9ab1e consensus state has no leader
I20260926 19:59:02.414928 12387 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: Sending DeleteTablet for 3 replicas of tablet fd98f43d7000438da50cc5285a4395ae
I20260926 19:59:02.416461 12387 catalog_manager.cc:6934] ReplaceTablet: tablet fd98f43d7000438da50cc5285a4395ae deleted and replaced by tablet 1e64bfd2fd6d436980b6f2c0bbee7c1b
I20260926 19:59:02.418322 12896 tablet_service.cc:1558] Processing DeleteTablet for tablet fd98f43d7000438da50cc5285a4395ae with delete_type TABLET_DATA_DELETED (replaced by tablet 1e64bfd2fd6d436980b6f2c0bbee7c1b) from {username='slave'} at 127.0.0.1:33594
W20260926 19:59:02.380201 12393 rpcz_store.cc:269] 0926 19:59:01.333050 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.333177 (+ 127us) service_pool.cc:224] Handling call
0926 19:59:01.552643 (+219466us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc85c0 00007fb36ac676fa 00007fb36c9d8d8c 00007fb36cad701e 00007fb36a67bfb4 00007fb36a67bf42 00007fb36a67be71 00007fb36a67bdec 00007fb36a67b0a2 00007fb36892d09c 00007fb36892c4d6
0926 19:59:02.378206 (+825563us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:59:02.350958 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:59:02.353595 (+ 2637us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:59:02.353619 (+ 24us) write_op.cc:435] Acquired schema lock
0926 19:59:02.353630 (+ 11us) tablet.cc:662] Decoding operations
0926 19:59:02.353884 (+ 254us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:59:02.353923 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0926 19:59:02.353945 (+ 22us) tablet.cc:685] Acquiring locks for 1 operations
0926 19:59:02.354087 (+ 142us) tablet.cc:701] Row locks acquired
0926 19:59:02.354104 (+ 17us) write_op.cc:260] PREPARE: finished
0926 19:59:02.354202 (+ 98us) write_op.cc:270] Start()
0926 19:59:02.354277 (+ 75us) write_op.cc:276] Timestamp: P: 1790452742354144 usec, L: 0
0926 19:59:02.354291 (+ 14us) op_driver.cc:348] REPLICATION: starting
0926 19:59:02.357202 (+ 2911us) log.cc:844] Serialized 356 byte log entry
0926 19:59:02.357782 (+ 580us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5400030554. stack: 00007fb363aca323 00007fb363aca220 00007fb36366e574 0000000000516964 00000000005168f9 00007fb36abc7aa1 00007fb36ac5a5e8 00007fb36b0ea791 00007fb36b0e9f68 00007fb36b0eec01 00007fb36b0eebb9 00007fb36b0eeb49 00007fb36b0eeb11 00007fb36b0edd3d 00000000005bd4e4 00000000005bd319
0926 19:59:02.367648 (+ 9866us) op_driver.cc:464] REPLICATION: finished
0926 19:59:02.368014 (+ 366us) write_op.cc:301] APPLY: starting
0926 19:59:02.368068 (+ 54us) tablet.cc:1394] starting BulkCheckPresence
0926 19:59:02.368159 (+ 91us) tablet.cc:1397] finished BulkCheckPresence
0926 19:59:02.368178 (+ 19us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:59:02.368340 (+ 162us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:59:02.368463 (+ 123us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0926 19:59:02.368505 (+ 42us) write_op.cc:312] APPLY: finished
0926 19:59:02.368695 (+ 190us) log.cc:844] Serialized 23 byte log entry
0926 19:59:02.368823 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:59:02.368894 (+ 71us) write_op.cc:454] Released schema lock
0926 19:59:02.368933 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0926 19:59:02.369134 (+ 201us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:59:02.369157 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":498765184,"child_traces":[["op",{"apply.queue_time_us":232,"apply.run_cpu_time_us":1097,"apply.run_wall_time_us":1096,"num_ops":1,"prepare.queue_time_us":778954,"prepare.run_cpu_time_us":2569,"prepare.run_wall_time_us":18076,"raft.queue_time_us":3322,"raft.run_cpu_time_us":778,"raft.run_wall_time_us":7510,"replication_time_us":13269,"spinlock_wait_cycles":5513088}]]}
W20260926 19:59:02.396960 12568 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s user 0.009s sys 0.007s
W20260926 19:59:02.436573 12594 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s user 0.011s sys 0.006s
W20260926 19:59:02.437011 12594 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s user 0.012s sys 0.006s
I20260926 19:59:02.481302 13013 consensus_queue.cc:260] T fe567f5d4c8c41ff9d681ca9ad3970e8 P f6e9d6439fd54b28925a68a6145d3dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:59:02.431834 12591 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s user 0.003s sys 0.001s
W20260926 19:59:02.430310 12590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s user 0.005s sys 0.010s
I20260926 19:59:02.432037 14002 log.cc:1199] T 700d05149a934d5e8bec9cc41873ad3e P f6e9d6439fd54b28925a68a6145d3dee: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/700d05149a934d5e8bec9cc41873ad3e
I20260926 19:59:02.515048 12886 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 341) took 1152 ms. Trace:
I20260926 19:59:02.515230 12887 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 338) took 1186 ms. Trace:
I20260926 19:59:02.515582 12887 rpcz_store.cc:276] 0926 19:59:01.329153 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.332236 (+ 3083us) service_pool.cc:224] Handling call
0926 19:59:01.333463 (+ 1227us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.333560 (+ 97us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.871598 (+538038us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.514780 (+643182us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.515174 (+ 394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.517077 12893 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 337) took 1194 ms. Trace:
I20260926 19:59:02.517447 12893 rpcz_store.cc:276] 0926 19:59:01.323026 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.328899 (+ 5873us) service_pool.cc:224] Handling call
0926 19:59:01.335274 (+ 6375us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.335341 (+ 67us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.850952 (+515611us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.516599 (+665647us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.517021 (+ 422us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":542820}
I20260926 19:59:02.520797 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 290) took 5394 ms. Trace:
I20260926 19:59:02.522300 12888 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33594 (request call id 350) took 1062 ms. Trace:
I20260926 19:59:02.522686 12888 rpcz_store.cc:276] 0926 19:59:01.460273 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.464504 (+ 4231us) service_pool.cc:224] Handling call
0926 19:59:01.585966 (+121462us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.586068 (+ 102us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.861781 (+275713us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.514498 (+652717us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.522234 (+ 7736us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":594197}
I20260926 19:59:02.515455 12886 rpcz_store.cc:276] 0926 19:59:01.362815 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.383314 (+ 20499us) service_pool.cc:224] Handling call
0926 19:59:01.415258 (+ 31944us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.415351 (+ 93us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.865754 (+450403us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.514522 (+648768us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.514973 (+ 451us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":571788}
I20260926 19:59:02.521152 13403 rpcz_store.cc:276] 0926 19:58:57.125878 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.135135 (+ 9257us) service_pool.cc:224] Handling call
0926 19:59:01.895792 (+4760657us) ts_tablet_manager.cc:1151] Deleting tablet acdb4bd8cd58444cb1d6d4b5610de20d
0926 19:59:01.895834 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:02.044977 (+149143us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.140559 (+ 95582us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.520731 (+380172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":351055,"tablet-delete.queue_time_us":4760073}
I20260926 19:59:02.526500 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 291) took 5390 ms. Trace:
I20260926 19:59:02.527029 13403 rpcz_store.cc:276] 0926 19:58:57.136051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.138511 (+ 2460us) service_pool.cc:224] Handling call
0926 19:59:02.525457 (+5386946us) ts_tablet_manager.cc:1151] Deleting tablet a26e6b837f67445c9383599b7e52de7b
0926 19:59:02.525499 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:02.526388 (+ 889us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5370840}
I20260926 19:59:02.528990 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet acdb4bd8cd58444cb1d6d4b5610de20d (table default.Table_309601918a5743ab8547d309642f118e [id=ba90bddd89d9494dbdc3d73d1e205aac]) successfully deleted
I20260926 19:59:02.529189 13403 tablet_replica.cc:333] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
W20260926 19:59:02.529726 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet a26e6b837f67445c9383599b7e52de7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a26e6b837f67445c9383599b7e52de7b already in progress: creating tablet
I20260926 19:59:02.529914 13403 raft_consensus.cc:2243] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:59:02.530459 13403 raft_consensus.cc:2272] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:02.532711 13403 ts_tablet_manager.cc:1916] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:02.532853 12886 tablet_service.cc:1511] Processing CreateTablet for tablet 41976314ad5d436f8e197a79dd2d954b (DEFAULT_TABLE table=default.Table_50f95c20537b40ed86ee75b4619cd35d [id=783c835dcee5492cbd87ccbd8449fe8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.535631 12886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41976314ad5d436f8e197a79dd2d954b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.544598 12888 tablet_service.cc:1511] Processing CreateTablet for tablet cf77f4b800674eaa9604ef789fb45ab8 (DEFAULT_TABLE table=default.Table_67a242744ca1463daaee9695a204dadd [id=17cee8cdd66f44a1a7013375cad8c496]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.533255 13092 raft_consensus.cc:359] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:02.552556 13092 raft_consensus.cc:385] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:02.552896 13092 raft_consensus.cc:740] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:59:02.549315 12888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf77f4b800674eaa9604ef789fb45ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.553941 13092 consensus_queue.cc:260] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:02.564697 12893 tablet_service.cc:1511] Processing CreateTablet for tablet 27f778b4927b4457b00b16972c067528 (DEFAULT_TABLE table=default.Table_ff2fc79b85024246be7c9dfd4c602925 [id=a28e2a2836654b8aae5753fd06c2e0e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.565904 12893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27f778b4927b4457b00b16972c067528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.588336 12394 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet fcad0bbb2bb44338bd56f2547e70ab9c (table default.Table_e8b7e8464d3d4efda2843dfb2dff6da1 [id=9908f61c35834c3e85cebb19da71fc49]). Expected version 1 got 0
I20260926 19:59:02.588567 13092 ts_tablet_manager.cc:1434] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.225s user 0.005s sys 0.001s
I20260926 19:59:02.553218 12199 master-stress-test.cc:581] Killed test thread 2/10
I20260926 19:59:02.597884 12199 master-stress-test.cc:579] Killing test thread 3/10
I20260926 19:59:02.598165 12199 master-stress-test.cc:581] Killed test thread 3/10
I20260926 19:59:02.598367 12199 master-stress-test.cc:579] Killing test thread 4/10
I20260926 19:59:02.598625 12199 master-stress-test.cc:581] Killed test thread 4/10
I20260926 19:59:02.598824 12199 master-stress-test.cc:579] Killing test thread 5/10
I20260926 19:59:02.599082 12199 master-stress-test.cc:581] Killed test thread 5/10
I20260926 19:59:02.599293 12199 master-stress-test.cc:579] Killing test thread 6/10
I20260926 19:59:02.599555 12199 master-stress-test.cc:581] Killed test thread 6/10
I20260926 19:59:02.599767 12199 master-stress-test.cc:579] Killing test thread 7/10
I20260926 19:59:02.600307 12199 master-stress-test.cc:581] Killed test thread 7/10
I20260926 19:59:02.600548 12199 master-stress-test.cc:579] Killing test thread 8/10
I20260926 19:59:02.600859 12199 master-stress-test.cc:581] Killed test thread 8/10
I20260926 19:59:02.601068 12199 master-stress-test.cc:579] Killing test thread 9/10
I20260926 19:59:02.601387 12199 master-stress-test.cc:581] Killed test thread 9/10
I20260926 19:59:02.601886 12199 master-stress-test.cc:579] Killing test thread 10/10
I20260926 19:59:02.602258 12199 master-stress-test.cc:581] Killed test thread 10/10
I20260926 19:59:02.602703 12199 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu with pid 12481
W20260926 19:59:02.061956 12205 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.11.233.193:46439, user_credentials={real_user=slave}} blocked reactor thread for 116962us
I20260926 19:59:02.608112 12896 tablet_service.cc:1511] Processing CreateTablet for tablet 1292b2fe4df541dfb6af7c0e6e2adab2 (DEFAULT_TABLE table=default.Table_51e292b5ade74377a0aa791835ab0ebe [id=121079ee19bb462c8c09cc7e37eec5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.609298 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1292b2fe4df541dfb6af7c0e6e2adab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.614265 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 33cd3ae497bb4e4e99c8b63fcfe2014d (DEFAULT_TABLE table=default.Table_1ef033f88cc24cfb911236e339e7ea2d [id=ef995d69fa9f47548715c66e436f17bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.615345 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33cd3ae497bb4e4e99c8b63fcfe2014d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.617903 12394 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet 59d1955b12dd47899b8a226fe0e980ae (table default.Table_64c355e960994a9197d1f606cf6017a3 [id=4ac26cbd7204431a903947e5f17c09b7]). Expected version 4 got 3
I20260926 19:59:02.630254 12887 tablet_service.cc:1511] Processing CreateTablet for tablet 27d3181049244bdbbbc55d98c6961969 (DEFAULT_TABLE table=default.Table_b68ea29b70b74dfe8183c6af78288a8b [id=467c1af9d1ae42e2a1ddb9f5e14dd046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.631592 13403 ts_tablet_manager.cc:1929] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:02.632015 13403 log.cc:1199] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/22f268b4a1f7423f917731c6b62aab3e
I20260926 19:59:02.632771 13403 ts_tablet_manager.cc:1950] T 22f268b4a1f7423f917731c6b62aab3e P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:59:02.634745 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 292) took 5335 ms. Trace:
I20260926 19:59:02.631660 12887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27d3181049244bdbbbc55d98c6961969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.647539 12891 tablet_service.cc:1558] Processing DeleteTablet for tablet acdb4bd8cd58444cb1d6d4b5610de20d with delete_type TABLET_DATA_DELETED (replaced by tablet 7645307e0999482bbe3329c8d61cc567) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:02.677014 13092 tablet_bootstrap.cc:492] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:59:02.683949 13092 tablet_bootstrap.cc:654] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:02.688874 12893 tablet_service.cc:1511] Processing CreateTablet for tablet 74cb427c28aa4c00970da28ee1b9a244 (DEFAULT_TABLE table=default.Table_c3fe6fcf32d349ad852cc8a8b766c175 [id=b71e559244894212939d8499a52cfe06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.690472 12893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74cb427c28aa4c00970da28ee1b9a244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.692202 12886 tablet_service.cc:1511] Processing CreateTablet for tablet 13899cad775b4c218a7a84860973ccea (DEFAULT_TABLE table=default.Table_fc73e336739940449da359776b1f4c50 [id=6850dabc2a8549e1842f432b9d9fd0e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.693347 12886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13899cad775b4c218a7a84860973ccea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.688513 13403 rpcz_store.cc:276] 0926 19:58:57.299311 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.299436 (+ 125us) service_pool.cc:224] Handling call
0926 19:59:02.528751 (+5229315us) ts_tablet_manager.cc:1151] Deleting tablet 22f268b4a1f7423f917731c6b62aab3e
0926 19:59:02.529086 (+ 335us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:02.555907 (+ 26821us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.631468 (+ 75561us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.634690 (+ 3222us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5228743}
I20260926 19:59:02.694589 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 293) took 5191 ms. Trace:
I20260926 19:59:02.694936 13403 rpcz_store.cc:276] 0926 19:58:57.502741 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.502872 (+ 131us) service_pool.cc:224] Handling call
0926 19:59:02.694236 (+5191364us) ts_tablet_manager.cc:1151] Deleting tablet 9411d3bca81641a4b3dfaffb2dc2bb51
0926 19:59:02.694282 (+ 46us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:02.694533 (+ 251us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5033098}
I20260926 19:59:02.696101 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 295) took 5030 ms. Trace:
I20260926 19:59:02.696425 13403 rpcz_store.cc:276] 0926 19:58:57.665907 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.666026 (+ 119us) service_pool.cc:224] Handling call
0926 19:59:02.695766 (+5029740us) ts_tablet_manager.cc:1151] Deleting tablet cfea8dbdd1ba49a5bd9d17652eb271c3
0926 19:59:02.695806 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:02.696051 (+ 245us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5029163}
I20260926 19:59:02.698196 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 118ec25b8c574ace8cfaa3e91fae8f95 (DEFAULT_TABLE table=default.Table_ec974c84b69a46b3a8820e70e43e7927 [id=f7d30fa2f4bd4426aebce7e08e1e051f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.698866 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 22f268b4a1f7423f917731c6b62aab3e (table default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]) successfully deleted
I20260926 19:59:02.699231 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118ec25b8c574ace8cfaa3e91fae8f95. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:02.699585 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 9411d3bca81641a4b3dfaffb2dc2bb51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9411d3bca81641a4b3dfaffb2dc2bb51 already in progress: creating tablet
W20260926 19:59:02.700266 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet cfea8dbdd1ba49a5bd9d17652eb271c3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cfea8dbdd1ba49a5bd9d17652eb271c3 already in progress: creating tablet
I20260926 19:59:02.721868 13092 tablet_bootstrap.cc:492] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:59:02.722412 13092 ts_tablet_manager.cc:1403] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.046s user 0.012s sys 0.005s
I20260926 19:59:02.697396 13403 tablet_replica.cc:333] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:59:02.724606 13403 raft_consensus.cc:2243] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:59:02.726224 13403 pending_rounds.cc:70] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Trying to abort 1 pending ops.
I20260926 19:59:02.726497 13403 pending_rounds.cc:77] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694417825370112 op_type: NO_OP noop_request { }
I20260926 19:59:02.726837 13403 raft_consensus.cc:2889] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:59:02.727175 13403 raft_consensus.cc:2272] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:02.729043 13403 ts_tablet_manager.cc:1916] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:02.725049 13092 raft_consensus.cc:359] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:02.729647 13092 raft_consensus.cc:385] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:02.729928 13092 raft_consensus.cc:740] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:59:02.730618 13092 consensus_queue.cc:260] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:02.737627 13092 ts_tablet_manager.cc:1434] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260926 19:59:02.741309 12886 tablet_service.cc:1511] Processing CreateTablet for tablet 50df02bfe05548dfb7f166adfa39256e (DEFAULT_TABLE table=default.Table_2627a2020d2043c1a7553b82baeeed59 [id=cf594f85a61e47d08c0d86182c8f7d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.749734 12896 tablet_service.cc:1511] Processing CreateTablet for tablet a6d74658dea845a3af17a79f06c444c5 (DEFAULT_TABLE table=default.Table_93b04d48f59c40bc9c1ab2ca57e0bef4 [id=04cab2faa680475b930f30c0b006efe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.754365 12886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50df02bfe05548dfb7f166adfa39256e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.760746 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6d74658dea845a3af17a79f06c444c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.761575 12888 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fd4c88f9304f19883c3e5da8f937b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 9599f427a64f40d7b185b23a849b46f3) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:02.765198 14020 raft_consensus.cc:493] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:02.765786 14020 raft_consensus.cc:515] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:02.767940 14020 leader_election.cc:290] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:02.767968 12724 rpcz_store.cc:276] 0926 19:58:59.421659 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.435850 (+ 14191us) service_pool.cc:224] Handling call
0926 19:58:59.715600 (+279750us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de7ce61 00007f2c9de5a0f3 00007f2c9de08640 00007f2c9c392bf4 00007f2c9c392b82 00007f2c9c392ab1
0926 19:58:59.715726 (+ 126us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.715810 (+ 84us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.299579 (+2583769us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.363720 (+ 64141us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.364110 (+ 390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64926464}
W20260926 19:59:02.259917 12696 leader_election.cc:343] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:02.443464 12726 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 331) took 3020 ms. Trace:
I20260926 19:59:02.546310 12741 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 363) took 1407 ms. Trace:
W20260926 19:59:02.771104 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 510983us
I20260926 19:59:02.771584 12741 rpcz_store.cc:276] 0926 19:59:01.138670 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.718594 (+579924us) service_pool.cc:224] Handling call
0926 19:59:01.889272 (+170678us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.889380 (+ 108us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.411031 (+521651us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.545873 (+134842us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.546250 (+ 377us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260926 19:59:02.772954 12858 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260926 19:59:02.773861 12694 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.196s user 0.009s sys 0.008s
I20260926 19:59:02.425081 13012 tablet_bootstrap.cc:492] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9: No bootstrap required, opened a new log
I20260926 19:59:02.775869 13092 tablet_bootstrap.cc:492] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:59:02.810257 14020 raft_consensus.cc:493] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:02.810722 14020 raft_consensus.cc:515] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:02.812839 14020 leader_election.cc:290] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:02.505623 13569 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38310 (request call id 323) took 3084 ms. Trace:
I20260926 19:59:02.818620 13012 ts_tablet_manager.cc:1403] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent bootstrapping tablet: real 1.893s user 0.010s sys 0.013s
I20260926 19:59:02.500873 12727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 330) took 3078 ms. Trace:
I20260926 19:59:02.409389 14030 raft_consensus.cc:515] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:02.401187 12729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 346) took 1819 ms. Trace:
I20260926 19:59:02.425220 12730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 355) took 1488 ms. Trace:
I20260926 19:59:02.452693 12731 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 354) took 1520 ms. Trace:
I20260926 19:59:02.565274 12733 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 364) took 1426 ms. Trace:
I20260926 19:59:02.838346 12733 rpcz_store.cc:276] 0926 19:59:01.138915 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.995803 (+856888us) service_pool.cc:224] Handling call
0926 19:59:01.998743 (+ 2940us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.998825 (+ 82us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.417260 (+418435us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.564818 (+147558us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.565213 (+ 395us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.560376 12734 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 361) took 1505 ms. Trace:
I20260926 19:59:02.846326 13092 tablet_bootstrap.cc:654] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:02.854023 12899 tablet_service.cc:1511] Processing CreateTablet for tablet fc5fd284545346cebf6cdd94cddb9136 (DEFAULT_TABLE table=default.Table_beef77b0962d44d8948a5f4984c0956f [id=35a95e8c58c64b73b6a80d93a709d475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.855054 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc5fd284545346cebf6cdd94cddb9136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.853977 12888 tablet_service.cc:1511] Processing CreateTablet for tablet b8b648d09e884eea9ad3c60bac045ad6 (DEFAULT_TABLE table=default.Table_f1b2f134d88540dab02e8debcbdbc755 [id=841d7e15064b4341a28277882b01b7f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.858297 12888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b648d09e884eea9ad3c60bac045ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.877561 12887 tablet_service.cc:1511] Processing CreateTablet for tablet f581f79c81214b859c4e9daad39c37f5 (DEFAULT_TABLE table=default.Table_5ccfc6e2d4a84071bc4376d8c425a135 [id=8b1941c0580d4bd598c0cb713013ebbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.878733 12887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f581f79c81214b859c4e9daad39c37f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.888414 13092 tablet_bootstrap.cc:492] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:59:02.888919 13092 ts_tablet_manager.cc:1403] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.113s user 0.010s sys 0.000s
I20260926 19:59:02.892015 13092 raft_consensus.cc:359] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
W20260926 19:59:02.896636 12857 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260926 19:59:02.892824 13092 raft_consensus.cc:385] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:02.898571 13092 raft_consensus.cc:740] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:59:02.435814 12735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 362) took 1381 ms. Trace:
I20260926 19:59:02.903083 13092 consensus_queue.cc:260] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:02.909792 12735 rpcz_store.cc:276] 0926 19:59:01.054913 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.710235 (+655322us) service_pool.cc:224] Handling call
0926 19:59:01.712915 (+ 2680us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.713005 (+ 90us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.337633 (+624628us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.435361 (+ 97728us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.435758 (+ 397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.566499 12736 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 359) took 1612 ms. Trace:
I20260926 19:59:02.910998 12736 rpcz_store.cc:276] 0926 19:59:00.954054 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.482496 (+528442us) service_pool.cc:224] Handling call
0926 19:59:01.693188 (+210692us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.693267 (+ 79us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.388795 (+695528us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.566065 (+177270us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.566440 (+ 375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.337334 13962 leader_election.cc:290] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901)
W20260926 19:59:02.798499 12694 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.221s user 0.009s sys 0.008s
I20260926 19:59:02.804649 12722 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 328) took 3382 ms. Trace:
I20260926 19:59:02.771124 12726 rpcz_store.cc:276] 0926 19:58:59.423128 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.431850 (+ 8722us) service_pool.cc:224] Handling call
0926 19:58:59.697225 (+265375us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de7ce61 00007f2c9de5a0f3 00007f2c9de08640 00007f2c9c392bf4 00007f2c9c392b82 00007f2c9c392ab1
0926 19:58:59.697333 (+ 108us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.697405 (+ 72us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.310216 (+2612811us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.442799 (+132583us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.443368 (+ 569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67000064}
W20260926 19:59:02.808697 12725 consensus_meta.cc:287] T 9ed22219df2b42ad86a6d7917ee0d990 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.501s user 0.001s sys 0.009s
I20260926 19:59:02.809405 14074 raft_consensus.cc:493] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8112c0b43d9c4e5389aa3a18f18b58a5)
I20260926 19:59:02.830657 12730 rpcz_store.cc:276] 0926 19:59:00.936774 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.952864 (+ 16090us) service_pool.cc:224] Handling call
0926 19:59:01.022291 (+ 69427us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.022399 (+ 108us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.019632 (+997233us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.424571 (+404939us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.425140 (+ 569us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.829499 12729 rpcz_store.cc:276] 0926 19:59:00.582041 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.815629 (+233588us) service_pool.cc:224] Handling call
0926 19:59:00.964627 (+148998us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:00.964710 (+ 83us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:01.445682 (+480972us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.400566 (+954884us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.401109 (+ 543us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.818125 13569 rpcz_store.cc:276] 0926 19:58:59.420758 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.429314 (+ 8556us) service_pool.cc:224] Handling call
0926 19:58:59.537802 (+108488us) ts_tablet_manager.cc:1151] Deleting tablet 22f268b4a1f7423f917731c6b62aab3e
0926 19:58:59.538410 (+ 608us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:01.704963 (+2166553us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:01.969936 (+264973us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.505536 (+535600us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":307993,"tablet-delete.queue_time_us":107358}
I20260926 19:59:02.836676 14030 leader_election.cc:290] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Requested vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:02.902998 12734 rpcz_store.cc:276] 0926 19:59:01.054575 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.570212 (+515637us) service_pool.cc:224] Handling call
0926 19:59:02.021034 (+450822us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:02.021111 (+ 77us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.446575 (+425464us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.559935 (+113360us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.560319 (+ 384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.915635 13092 ts_tablet_manager.cc:1434] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.026s user 0.004s sys 0.004s
I20260926 19:59:02.909432 13403 ts_tablet_manager.cc:1929] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:02.916044 13403 log.cc:1199] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/b32f8b0d480547a58ba5c6fed6483a74
I20260926 19:59:02.916795 13403 ts_tablet_manager.cc:1950] T b32f8b0d480547a58ba5c6fed6483a74 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:59:02.916877 13569 tablet_replica.cc:333] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9: stopping tablet replica
I20260926 19:59:02.916831 14074 raft_consensus.cc:515] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:59:02.918103 12725 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 332) took 3494 ms. Trace:
I20260926 19:59:02.918437 12725 rpcz_store.cc:276] 0926 19:58:59.423362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.433897 (+ 10535us) service_pool.cc:224] Handling call
0926 19:58:59.701879 (+267982us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de7ce61 00007f2c9de5a0f3 00007f2c9de08640 00007f2c9c392bf4 00007f2c9c392b82 00007f2c9c392ab1
0926 19:58:59.701983 (+ 104us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.702069 (+ 86us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.307045 (+2604976us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.916280 (+609235us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.918046 (+ 1766us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61049600}
I20260926 19:59:02.920984 14074 leader_election.cc:290] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:02.912660 12722 rpcz_store.cc:276] 0926 19:58:59.422243 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.439421 (+ 17178us) service_pool.cc:224] Handling call
0926 19:58:59.700083 (+260662us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de7ce61 00007f2c9de5a0f3 00007f2c9de08640 00007f2c9c392bf4 00007f2c9c392b82 00007f2c9c392ab1
0926 19:58:59.700183 (+ 100us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.700250 (+ 67us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:00.987187 (+1286937us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.339967 (+1352780us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.804400 (+464433us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41143680}
I20260926 19:59:02.836722 12731 rpcz_store.cc:276] 0926 19:59:00.932432 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.942693 (+ 10261us) service_pool.cc:224] Handling call
0926 19:59:01.026219 (+ 83526us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:01.026305 (+ 86us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.325081 (+1298776us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.452054 (+126973us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.452612 (+ 558us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:02.925797 12896 tablet_service.cc:1511] Processing CreateTablet for tablet 3bac488d42444d07962be0fe9e8ed2e5 (DEFAULT_TABLE table=default.Table_ba0bfeee8cac4c709fe66a3645e5b71e [id=041be739e2e34c6c9a0959203d8c1a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.825276 12727 rpcz_store.cc:276] 0926 19:58:59.422863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.429935 (+ 7072us) service_pool.cc:224] Handling call
0926 19:58:59.698808 (+268873us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de7ce61 00007f2c9de5a0f3 00007f2c9de08640 00007f2c9c392bf4 00007f2c9c392b82 00007f2c9c392ab1
0926 19:58:59.698937 (+ 129us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.699467 (+ 530us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.357983 (+2658516us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.500361 (+142378us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:02.500801 (+ 440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71347072}
I20260926 19:59:02.933096 14020 raft_consensus.cc:493] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ae87963ed5e4d92a896bf4a73af4da9)
I20260926 19:59:02.933640 14020 raft_consensus.cc:515] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:59:02.936050 14020 leader_election.cc:290] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:02.937549 12886 tablet_service.cc:1511] Processing CreateTablet for tablet 6c23ecdde3b04ce784f3f4bf0b2f0441 (DEFAULT_TABLE table=default.Table_b638b85037f24946b493b98629cab24e [id=b748bf49932b4e0db7de11beebb15173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.938409 12893 tablet_service.cc:1511] Processing CreateTablet for tablet 9424c4d822964f40b1feeb2764519d0d (DEFAULT_TABLE table=default.Table_8ec529fa6fe04ad1b44a473e8610375a [id=cfd92ee473eb4ffe9d696075b6c6d10c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.939523 12893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9424c4d822964f40b1feeb2764519d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.939640 13092 tablet_bootstrap.cc:492] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:59:02.949088 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 294) took 5375 ms. Trace:
I20260926 19:59:02.949568 13403 rpcz_store.cc:276] 0926 19:58:57.573237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.573351 (+ 114us) service_pool.cc:224] Handling call
0926 19:59:02.697253 (+5123902us) ts_tablet_manager.cc:1151] Deleting tablet b32f8b0d480547a58ba5c6fed6483a74
0926 19:59:02.697294 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:02.854949 (+157655us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.909311 (+ 54362us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:02.949009 (+ 39698us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5027550}
I20260926 19:59:02.952301 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7645307e0999482bbe3329c8d61cc567" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:02.955610 12756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "f6e9d6439fd54b28925a68a6145d3dee" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:02.961665 14100 raft_consensus.cc:493] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f6e9d6439fd54b28925a68a6145d3dee)
I20260926 19:59:02.962261 14100 raft_consensus.cc:515] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:59:02.965178 13092 tablet_bootstrap.cc:654] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:02.981470 12891 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c7d0435f3564cb38b6cbca713009c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:58:52 UTC) from {username='slave'} at 127.0.0.1:33594
I20260926 19:59:02.984606 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 296) took 5308 ms. Trace:
I20260926 19:59:02.984968 13403 rpcz_store.cc:276] 0926 19:58:57.675791 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:57.675924 (+ 133us) service_pool.cc:224] Handling call
0926 19:59:02.962499 (+5286575us) ts_tablet_manager.cc:1151] Deleting tablet 54553d20d48148ae88fa4cc2ce021779
0926 19:59:02.984295 (+ 21796us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:02.984548 (+ 253us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5285972}
I20260926 19:59:02.986429 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 297) took 4940 ms. Trace:
W20260926 19:59:02.986821 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 54553d20d48148ae88fa4cc2ce021779 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54553d20d48148ae88fa4cc2ce021779 already in progress: creating tablet
I20260926 19:59:02.986718 13403 rpcz_store.cc:276] 0926 19:58:58.046242 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:58.046375 (+ 133us) service_pool.cc:224] Handling call
0926 19:59:02.986138 (+4939763us) ts_tablet_manager.cc:1151] Deleting tablet e9c7b63db44742108dc132df5089e0d9
0926 19:59:02.986174 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:02.986382 (+ 208us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4939177}
I20260926 19:59:02.987887 13403 tablet_replica.cc:333] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
W20260926 19:59:02.987823 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet e9c7b63db44742108dc132df5089e0d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9c7b63db44742108dc132df5089e0d9 already in progress: creating tablet
I20260926 19:59:02.988647 13403 raft_consensus.cc:2243] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:02.989761 12891 tablet_service.cc:1511] Processing CreateTablet for tablet d0f242a9fb2f4fc786b05d5336473cbc (DEFAULT_TABLE table=default.Table_4ae8526c5c1f43859924b2e27dbbc75a [id=59951d3d9e864fcbbc2766139849a3fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:02.990144 12886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c23ecdde3b04ce784f3f4bf0b2f0441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.990792 12891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0f242a9fb2f4fc786b05d5336473cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:02.999794 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bac488d42444d07962be0fe9e8ed2e5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:03.014711 12723 consensus_meta.cc:287] T a0c465b503b54e569fbf529da3ac316c P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent flushing consensus metadata: real 0.664s user 0.006s sys 0.007s
I20260926 19:59:03.014961 12737 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 365) took 1548 ms. Trace:
I20260926 19:59:02.950474 14074 raft_consensus.cc:493] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:03.015353 12737 rpcz_store.cc:276] 0926 19:59:01.466077 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:02.012158 (+546081us) service_pool.cc:224] Handling call
0926 19:59:02.013864 (+ 1706us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:59:02.013950 (+ 86us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.545042 (+531092us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.013749 (+468707us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:03.014736 (+ 987us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260926 19:59:03.017727 14074 raft_consensus.cc:515] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:03.018381 12723 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:38310 (request call id 327) took 3596 ms. Trace:
I20260926 19:59:03.018718 12723 rpcz_store.cc:276] 0926 19:58:59.421949 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.437415 (+ 15466us) service_pool.cc:224] Handling call
0926 19:58:59.698013 (+260598us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b5800000f98. stack: 00007f2c95c18323 00007f2c95c18220 00007f2c957bc574 0000000000516964 00000000005168f9 00007f2c95ace8a1 00007f2c95c12b51 00007f2c95c12a02 00007f2c9eb5d119 00007f2c9dd97a70 00007f2c9de7ce61 00007f2c9de5a0f3 00007f2c9de08640 00007f2c9c392bf4 00007f2c9c392b82 00007f2c9c392ab1
0926 19:58:59.698123 (+ 110us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0926 19:58:59.698205 (+ 82us) ts_tablet_manager.cc:682] Creating new metadata...
0926 19:59:02.350244 (+2652039us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.017847 (+667603us) tablet_replica.cc:206] Creating consensus instance
0926 19:59:03.018328 (+ 481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52452608}
I20260926 19:59:03.019846 14074 leader_election.cc:290] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:03.021942 14100 leader_election.cc:290] T 8ff071a16f1244cebc8c93e5508280a3 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:03.026496 12919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7d0435f3564cb38b6cbca713009c2e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:59:03.027149 12919 raft_consensus.cc:3060] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:59:03.031157 12857 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260926 19:59:03.031409 12924 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "700d05149a934d5e8bec9cc41873ad3e" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:59:03.031920 12924 raft_consensus.cc:3060] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:03.032884 12887 tablet_service.cc:1511] Processing CreateTablet for tablet d3664f5cc8f44666aa9c157cfcdb329d (DEFAULT_TABLE table=default.Table_33ac50d900914f2982f1880450cd2b5e [id=ad4f072144324740ac0345efc517390f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.033279 12731 tablet_service.cc:1558] Processing DeleteTablet for tablet a1e776237ca24a32a8e73d2f0628df3e with delete_type TABLET_DATA_DELETED (replaced by tablet 50d6cbf3191d4043b79f421a26d5f5ec) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:03.034061 12722 tablet_service.cc:1558] Processing DeleteTablet for tablet fd98f43d7000438da50cc5285a4395ae with delete_type TABLET_DATA_DELETED (replaced by tablet 1e64bfd2fd6d436980b6f2c0bbee7c1b) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:03.034513 12923 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb035b50e8f445ba492e67796085123" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5"
I20260926 19:59:03.034966 12923 raft_consensus.cc:3060] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:03.026307 12725 tablet_service.cc:1558] Processing DeleteTablet for tablet c4d86fa6b40b4fc49c4fa67b63792831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:01 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:02.989455 13403 pending_rounds.cc:70] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Trying to abort 1 pending ops.
I20260926 19:59:03.042851 13403 pending_rounds.cc:77] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333694417792212992 op_type: NO_OP noop_request { }
I20260926 19:59:03.043164 13403 raft_consensus.cc:2889] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:59:03.043474 13403 raft_consensus.cc:2272] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:03.048947 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 13899cad775b4c218a7a84860973ccea (DEFAULT_TABLE table=default.Table_fc73e336739940449da359776b1f4c50 [id=6850dabc2a8549e1842f432b9d9fd0e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.049211 12729 tablet_service.cc:1511] Processing CreateTablet for tablet 41976314ad5d436f8e197a79dd2d954b (DEFAULT_TABLE table=default.Table_50f95c20537b40ed86ee75b4619cd35d [id=783c835dcee5492cbd87ccbd8449fe8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.050355 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13899cad775b4c218a7a84860973ccea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.053980 12923 raft_consensus.cc:2468] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:59:03.055478 12737 tablet_service.cc:1511] Processing CreateTablet for tablet 33cd3ae497bb4e4e99c8b63fcfe2014d (DEFAULT_TABLE table=default.Table_1ef033f88cc24cfb911236e339e7ea2d [id=ef995d69fa9f47548715c66e436f17bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.047920 12734 tablet_service.cc:1511] Processing CreateTablet for tablet c1ec11f9e2f546d0b0ce2b1bc3cc49a5 (DEFAULT_TABLE table=default.Table_eca0d16a6176401c9021c2153e65b550 [id=bd4ee4e9347b4e5bad7c88dc23e151d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.052173 12733 tablet_service.cc:1511] Processing CreateTablet for tablet 27d3181049244bdbbbc55d98c6961969 (DEFAULT_TABLE table=default.Table_b68ea29b70b74dfe8183c6af78288a8b [id=467c1af9d1ae42e2a1ddb9f5e14dd046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.058394 12728 tablet_service.cc:1511] Processing CreateTablet for tablet f581f79c81214b859c4e9daad39c37f5 (DEFAULT_TABLE table=default.Table_5ccfc6e2d4a84071bc4376d8c425a135 [id=8b1941c0580d4bd598c0cb713013ebbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.059142 12729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41976314ad5d436f8e197a79dd2d954b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.059031 12739 tablet_service.cc:1511] Processing CreateTablet for tablet b8b648d09e884eea9ad3c60bac045ad6 (DEFAULT_TABLE table=default.Table_f1b2f134d88540dab02e8debcbdbc755 [id=841d7e15064b4341a28277882b01b7f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.060212 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b648d09e884eea9ad3c60bac045ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.039188 14109 tablet.cc:1751] T 552d11cb23144691809c554c125cad0e P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:59:03.061872 12759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "700d05149a934d5e8bec9cc41873ad3e" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.062706 12759 raft_consensus.cc:2393] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in current term 1: Already voted for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in this term.
I20260926 19:59:03.046864 12735 tablet_service.cc:1511] Processing CreateTablet for tablet 27f778b4927b4457b00b16972c067528 (DEFAULT_TABLE table=default.Table_ff2fc79b85024246be7c9dfd4c602925 [id=a28e2a2836654b8aae5753fd06c2e0e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.051568 12736 tablet_service.cc:1511] Processing CreateTablet for tablet cf77f4b800674eaa9604ef789fb45ab8 (DEFAULT_TABLE table=default.Table_67a242744ca1463daaee9695a204dadd [id=17cee8cdd66f44a1a7013375cad8c496]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.068141 14093 raft_consensus.cc:493] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8112c0b43d9c4e5389aa3a18f18b58a5)
I20260926 19:59:03.068486 12733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27d3181049244bdbbbc55d98c6961969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.069794 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33cd3ae497bb4e4e99c8b63fcfe2014d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.062697 12732 tablet_service.cc:1511] Processing CreateTablet for tablet 3bac488d42444d07962be0fe9e8ed2e5 (DEFAULT_TABLE table=default.Table_ba0bfeee8cac4c709fe66a3645e5b71e [id=041be739e2e34c6c9a0959203d8c1a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.067167 13012 raft_consensus.cc:359] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:03.072122 13012 raft_consensus.cc:385] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:03.074972 13012 raft_consensus.cc:740] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Initialized, Role: FOLLOWER
I20260926 19:59:03.064173 12726 tablet_service.cc:1558] Processing DeleteTablet for tablet 986a28107bd54199bf2cda96978150c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:59:01 UTC) from {username='slave'} at 127.0.0.1:38310
I20260926 19:59:03.082765 12734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1ec11f9e2f546d0b0ce2b1bc3cc49a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.068724 14093 raft_consensus.cc:515] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:59:03.077798 13012 consensus_queue.cc:260] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:03.071714 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62e75eb4b3e8493b9d599c1c4ad4c67f" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:03.057837 12919 raft_consensus.cc:2468] T 8c7d0435f3564cb38b6cbca713009c2e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:59:03.087414 12924 raft_consensus.cc:2468] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 1.
I20260926 19:59:03.097206 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27f778b4927b4457b00b16972c067528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.098667 13092 tablet_bootstrap.cc:492] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:59:03.099175 13092 ts_tablet_manager.cc:1403] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.160s user 0.010s sys 0.001s
I20260926 19:59:03.102716 13092 raft_consensus.cc:359] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:03.116670 14112 raft_consensus.cc:493] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:59:03.121325 13092 raft_consensus.cc:385] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:59:03.108035 12860 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260926 19:59:03.109819 12887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3664f5cc8f44666aa9c157cfcdb329d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.119299 14112 raft_consensus.cc:515] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
I20260926 19:59:03.126905 13092 raft_consensus.cc:740] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:59:03.140066 14093 leader_election.cc:290] T 99465590a0e64b35b8a8f07e8777b524 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:03.143204 12738 tablet_service.cc:1511] Processing CreateTablet for tablet d0f242a9fb2f4fc786b05d5336473cbc (DEFAULT_TABLE table=default.Table_4ae8526c5c1f43859924b2e27dbbc75a [id=59951d3d9e864fcbbc2766139849a3fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.156728 14114 raft_consensus.cc:493] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ae87963ed5e4d92a896bf4a73af4da9)
I20260926 19:59:03.157312 14114 raft_consensus.cc:515] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } }
I20260926 19:59:03.136929 13092 consensus_queue.cc:260] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:03.160874 14114 leader_election.cc:290] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:03.181172 14020 raft_consensus.cc:493] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:03.181792 14020 raft_consensus.cc:515] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:03.184407 14020 leader_election.cc:290] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:03.085565 12722 tablet_service.cc:1511] Processing CreateTablet for tablet 118ec25b8c574ace8cfaa3e91fae8f95 (DEFAULT_TABLE table=default.Table_ec974c84b69a46b3a8820e70e43e7927 [id=f7d30fa2f4bd4426aebce7e08e1e051f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.198314 13403 ts_tablet_manager.cc:1916] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:03.177970 12731 tablet_service.cc:1511] Processing CreateTablet for tablet 74cb427c28aa4c00970da28ee1b9a244 (DEFAULT_TABLE table=default.Table_c3fe6fcf32d349ad852cc8a8b766c175 [id=b71e559244894212939d8499a52cfe06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.204164 12723 tablet_service.cc:1511] Processing CreateTablet for tablet 1292b2fe4df541dfb6af7c0e6e2adab2 (DEFAULT_TABLE table=default.Table_51e292b5ade74377a0aa791835ab0ebe [id=121079ee19bb462c8c09cc7e37eec5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.205724 12724 tablet_service.cc:1511] Processing CreateTablet for tablet fc5fd284545346cebf6cdd94cddb9136 (DEFAULT_TABLE table=default.Table_beef77b0962d44d8948a5f4984c0956f [id=35a95e8c58c64b73b6a80d93a709d475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.209272 13012 ts_tablet_manager.cc:1434] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9: Time spent starting tablet: real 0.390s user 0.004s sys 0.004s
I20260926 19:59:03.211635 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 512 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:59:03.212172 13092 ts_tablet_manager.cc:1434] T cc9f7a2e5d1c41a8ac793a31696fb311 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.113s user 0.008s sys 0.000s
I20260926 19:59:03.213158 13092 tablet_bootstrap.cc:492] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:59:03.207067 12728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f581f79c81214b859c4e9daad39c37f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.210608 14112 leader_election.cc:290] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers db2490adb6e84090b780f52470e580ab (127.11.233.253:37913), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:59:03.224453 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59f7dc92330d400f93f427f3f79ad826" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:59:03.225127 12761 raft_consensus.cc:3060] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:03.209646 12730 tablet_service.cc:1511] Processing CreateTablet for tablet d3664f5cc8f44666aa9c157cfcdb329d (DEFAULT_TABLE table=default.Table_33ac50d900914f2982f1880450cd2b5e [id=ad4f072144324740ac0345efc517390f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.232237 12757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad76fc959e24545bdfad19f988fce93" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.233444 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 6c23ecdde3b04ce784f3f4bf0b2f0441 (DEFAULT_TABLE table=default.Table_b638b85037f24946b493b98629cab24e [id=b748bf49932b4e0db7de11beebb15173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.193272 12726 tablet_service.cc:1511] Processing CreateTablet for tablet 9424c4d822964f40b1feeb2764519d0d (DEFAULT_TABLE table=default.Table_8ec529fa6fe04ad1b44a473e8610375a [id=cfd92ee473eb4ffe9d696075b6c6d10c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.246647 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa82c030ec34ac5a8c448d80cdbae95" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.269845 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fd4c88f9304f19883c3e5da8f937b5" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.270573 12751 raft_consensus.cc:2468] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:59:03.274276 12727 tablet_service.cc:1511] Processing CreateTablet for tablet a6d74658dea845a3af17a79f06c444c5 (DEFAULT_TABLE table=default.Table_93b04d48f59c40bc9c1ab2ca57e0bef4 [id=04cab2faa680475b930f30c0b006efe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.274461 12857 leader_election.cc:304] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:59:03.275548 12857 leader_election.cc:343] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Not found: Tablet not found: cd86f01d82d041f4852f83f2547e697f
I20260926 19:59:03.275911 12857 leader_election.cc:304] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:59:03.276445 12857 raft_consensus.cc:2686] T cd86f01d82d041f4852f83f2547e697f P 8112c0b43d9c4e5389aa3a18f18b58a5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 19:59:03.278126 14114 raft_consensus.cc:2749] T 700d05149a934d5e8bec9cc41873ad3e P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:03.278970 12917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fd4c88f9304f19883c3e5da8f937b5" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:03.279587 12917 raft_consensus.cc:2468] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae87963ed5e4d92a896bf4a73af4da9 in term 0.
I20260926 19:59:03.280535 12922 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99465590a0e64b35b8a8f07e8777b524" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:03.282150 13092 tablet_bootstrap.cc:654] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:03.281963 12921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ff071a16f1244cebc8c93e5508280a3" candidate_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" is_pre_election: true
I20260926 19:59:03.283020 12857 leader_election.cc:304] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:59:03.283707 14114 raft_consensus.cc:2804] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:03.284060 14114 raft_consensus.cc:493] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:03.284400 14114 raft_consensus.cc:3060] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:03.298811 14128 tablet.cc:1751] T 552d11cb23144691809c554c125cad0e P 8112c0b43d9c4e5389aa3a18f18b58a5: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260926 19:59:03.297875 12750 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986a28107bd54199bf2cda96978150c3" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.304214 12750 raft_consensus.cc:2468] T 986a28107bd54199bf2cda96978150c3 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 0.
I20260926 19:59:03.251937 12754 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a7d5e580f5645ab95fa03d5f669dc32" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.293375 14114 raft_consensus.cc:515] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:03.318552 14114 leader_election.cc:290] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461), f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439)
I20260926 19:59:03.325814 13569 raft_consensus.cc:2243] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:59:03.329798 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcad0bbb2bb44338bd56f2547e70ab9c" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.332321 13403 ts_tablet_manager.cc:1929] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:03.332736 13403 log.cc:1199] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/6f666e1a8d214172bf5b752e9ec374bd
I20260926 19:59:03.333523 13403 ts_tablet_manager.cc:1950] T 6f666e1a8d214172bf5b752e9ec374bd P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:59:03.335632 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 309) took 3881 ms. Trace:
I20260926 19:59:03.335945 13403 rpcz_store.cc:276] 0926 19:58:59.453856 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.453987 (+ 131us) service_pool.cc:224] Handling call
0926 19:59:02.987742 (+3533755us) ts_tablet_manager.cc:1151] Deleting tablet 6f666e1a8d214172bf5b752e9ec374bd
0926 19:59:02.987778 (+ 36us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.307250 (+319472us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.332188 (+ 24938us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.335571 (+ 3383us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3533142}
I20260926 19:59:03.337260 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 323) took 3586 ms. Trace:
I20260926 19:59:03.337596 13403 rpcz_store.cc:276] 0926 19:58:59.750558 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:58:59.758363 (+ 7805us) service_pool.cc:224] Handling call
0926 19:59:03.336918 (+3578555us) ts_tablet_manager.cc:1151] Deleting tablet 01d8740b03594023a8420c7a7121b80c
0926 19:59:03.336963 (+ 45us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.337206 (+ 243us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3577961}
W20260926 19:59:03.338389 12696 leader_election.cc:343] T b32f8b0d480547a58ba5c6fed6483a74 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Not found: Tablet not found: b32f8b0d480547a58ba5c6fed6483a74
I20260926 19:59:03.338732 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 322) took 3588 ms. Trace:
I20260926 19:59:03.339020 13403 rpcz_store.cc:276] 0926 19:58:59.750246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.010489 (+260243us) service_pool.cc:224] Handling call
0926 19:59:03.338428 (+3327939us) ts_tablet_manager.cc:1151] Deleting tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534
0926 19:59:03.338468 (+ 40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.338681 (+ 213us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3314775}
I20260926 19:59:03.340113 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 325) took 2453 ms. Trace:
I20260926 19:59:03.340408 13403 rpcz_store.cc:276] 0926 19:59:00.887183 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.887462 (+ 279us) service_pool.cc:224] Handling call
0926 19:59:03.339822 (+2452360us) ts_tablet_manager.cc:1151] Deleting tablet f28394cd29c44af6b8c662f6c9e1165d
0926 19:59:03.339860 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.340069 (+ 209us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2451776}
I20260926 19:59:03.341524 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 324) took 2460 ms. Trace:
I20260926 19:59:03.341497 12696 leader_election.cc:304] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:59:03.341820 13403 rpcz_store.cc:276] 0926 19:59:00.880803 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.881724 (+ 921us) service_pool.cc:224] Handling call
0926 19:59:03.341200 (+2459476us) ts_tablet_manager.cc:1151] Deleting tablet 9ed22219df2b42ad86a6d7917ee0d990
0926 19:59:03.341238 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.341454 (+ 216us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2450599}
I20260926 19:59:03.342893 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 328) took 2205 ms. Trace:
I20260926 19:59:03.343192 13403 rpcz_store.cc:276] 0926 19:59:01.137831 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.138006 (+ 175us) service_pool.cc:224] Handling call
0926 19:59:03.342599 (+2204593us) ts_tablet_manager.cc:1151] Deleting tablet 6a85852c08304e0fbbbcbb2c0d24b27e
0926 19:59:03.342637 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.342847 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2193930}
I20260926 19:59:03.344285 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 326) took 2438 ms. Trace:
I20260926 19:59:03.344594 13403 rpcz_store.cc:276] 0926 19:59:00.905762 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.929035 (+ 23273us) service_pool.cc:224] Handling call
0926 19:59:03.343992 (+2414957us) ts_tablet_manager.cc:1151] Deleting tablet 3488f0076b0f4ef19f7cf23a93cd20e3
0926 19:59:03.344029 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.344242 (+ 213us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2182394}
I20260926 19:59:03.345537 13403 tablet_replica.cc:333] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:59:03.346310 13403 raft_consensus.cc:2243] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:03.346832 13403 raft_consensus.cc:2272] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:03.349359 13403 ts_tablet_manager.cc:1916] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:03.333130 14119 tablet.cc:1751] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 3ae87963ed5e4d92a896bf4a73af4da9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260926 19:59:03.351402 12380 catalog_manager.cc:5063] TS 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): tablet 22f268b4a1f7423f917731c6b62aab3e (table default.Table_a9b6637851b849c38be655d4ad6e9a5c [id=56cf8721abca44da91e524278b6b1208]) successfully deleted
I20260926 19:59:03.331508 12761 raft_consensus.cc:2468] T 59f7dc92330d400f93f427f3f79ad826 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8112c0b43d9c4e5389aa3a18f18b58a5 in term 1.
I20260926 19:59:03.354866 13783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db9d871438324598b27dd67b31c017d9" candidate_term: 7 candidate_status { last_received { term: 6 index: 512 } } ignore_live_leader: false dest_uuid: "db2490adb6e84090b780f52470e580ab" is_pre_election: true
I20260926 19:59:03.357659 12227 leader_election.cc:304] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db9d871438324598b27dd67b31c017d9; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db2490adb6e84090b780f52470e580ab
I20260926 19:59:03.358537 14112 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db9d871438324598b27dd67b31c017d9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:03.365554 13403 ts_tablet_manager.cc:1929] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:03.365916 13403 log.cc:1199] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/2ad76fc959e24545bdfad19f988fce93
I20260926 19:59:03.366863 13403 ts_tablet_manager.cc:1950] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
I20260926 19:59:03.367977 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 330) took 2208 ms. Trace:
I20260926 19:59:03.368803 12752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb035b50e8f445ba492e67796085123" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.374255 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b7715b8fcb4b59917ae14a83f9955d" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.376518 12725 tablet_service.cc:1511] Processing CreateTablet for tablet 50df02bfe05548dfb7f166adfa39256e (DEFAULT_TABLE table=default.Table_2627a2020d2043c1a7553b82baeeed59 [id=cf594f85a61e47d08c0d86182c8f7d16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.389816 12696 leader_election.cc:304] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:59:03.405877 13569 raft_consensus.cc:2272] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:59:03.407392 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 46806.8us
I20260926 19:59:03.409754 13915 raft_consensus.cc:2804] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:03.410290 13915 raft_consensus.cc:697] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:59:03.411123 13915 consensus_queue.cc:237] T edb035b50e8f445ba492e67796085123 P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:03.414582 14031 raft_consensus.cc:2804] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:03.417682 14031 raft_consensus.cc:697] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:59:03.418511 14031 consensus_queue.cc:237] T 8c7d0435f3564cb38b6cbca713009c2e P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:03.422068 14109 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:38310 (request call id 340) took 3068 ms. Trace:
I20260926 19:59:03.423662 12696 leader_election.cc:304] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:59:03.426299 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf77f4b800674eaa9604ef789fb45ab8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.430078 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0f242a9fb2f4fc786b05d5336473cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.430850 12724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc5fd284545346cebf6cdd94cddb9136. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:03.435899 12857 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260926 19:59:03.439885 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 6f666e1a8d214172bf5b752e9ec374bd (table default.Table_c7d455485dd14d60806f7a48b45ce98b [id=453a40bc60fd4eeaa2ffc14d887320ec]) successfully deleted
W20260926 19:59:03.440775 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 01d8740b03594023a8420c7a7121b80c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 01d8740b03594023a8420c7a7121b80c already in progress: creating tablet
W20260926 19:59:03.441541 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7cd9c4ac9aa4e8da28f0fa28c4f2534 already in progress: creating tablet
W20260926 19:59:03.442255 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet f28394cd29c44af6b8c662f6c9e1165d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f28394cd29c44af6b8c662f6c9e1165d already in progress: creating tablet
I20260926 19:59:03.442545 14133 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:59:03.443071 14133 raft_consensus.cc:515] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9d871438324598b27dd67b31c017d9" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 33143 } } peers { permanent_uuid: "db2490adb6e84090b780f52470e580ab" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 37913 } } peers { permanent_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 44243 } }
W20260926 19:59:03.444806 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 9ed22219df2b42ad86a6d7917ee0d990 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ed22219df2b42ad86a6d7917ee0d990 already in progress: creating tablet
W20260926 19:59:03.445492 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 6a85852c08304e0fbbbcbb2c0d24b27e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a85852c08304e0fbbbcbb2c0d24b27e already in progress: creating tablet
W20260926 19:59:03.446126 12382 catalog_manager.cc:5043] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): delete failed for tablet 3488f0076b0f4ef19f7cf23a93cd20e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3488f0076b0f4ef19f7cf23a93cd20e3 already in progress: creating tablet
I20260926 19:59:03.450348 12730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3664f5cc8f44666aa9c157cfcdb329d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.452600 12248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 512 } } ignore_live_leader: false dest_uuid: "db9d871438324598b27dd67b31c017d9" is_pre_election: true
I20260926 19:59:03.453498 12406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db2490adb6e84090b780f52470e580ab" candidate_term: 7 candidate_status { last_received { term: 6 index: 512 } } ignore_live_leader: false dest_uuid: "8a69f5870f8d461aa5e61f6999bb05c6" is_pre_election: true
I20260926 19:59:03.455040 13739 leader_election.cc:304] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db2490adb6e84090b780f52470e580ab; no voters: 8a69f5870f8d461aa5e61f6999bb05c6, db9d871438324598b27dd67b31c017d9
I20260926 19:59:03.462229 14102 raft_consensus.cc:2804] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:03.463128 12722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118ec25b8c574ace8cfaa3e91fae8f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.465821 12723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1292b2fe4df541dfb6af7c0e6e2adab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.479404 13092 tablet_bootstrap.cc:492] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:59:03.480015 13092 ts_tablet_manager.cc:1403] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.267s user 0.007s sys 0.005s
I20260926 19:59:03.368269 13403 rpcz_store.cc:276] 0926 19:59:01.159528 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.159705 (+ 177us) service_pool.cc:224] Handling call
0926 19:59:03.345366 (+2185661us) ts_tablet_manager.cc:1151] Deleting tablet 2ad76fc959e24545bdfad19f988fce93
0926 19:59:03.345403 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.357438 (+ 12035us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.365392 (+ 7954us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.367909 (+ 2517us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2173387}
I20260926 19:59:03.454475 14133 leader_election.cc:290] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers db9d871438324598b27dd67b31c017d9 (127.11.233.254:33143), 8a69f5870f8d461aa5e61f6999bb05c6 (127.11.233.252:44243)
I20260926 19:59:03.491344 14133 raft_consensus.cc:2749] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:03.492443 12731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74cb427c28aa4c00970da28ee1b9a244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.496171 13092 raft_consensus.cc:359] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:03.497979 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 2ad76fc959e24545bdfad19f988fce93 (table default.Table_4ceba47368094646b52f05a4bd441a64 [id=6a6df456d54045ab95e8dc2c70856189]) successfully deleted
I20260926 19:59:03.498852 13403 tablet_replica.cc:333] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:59:03.500886 13403 raft_consensus.cc:2243] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:03.501565 13403 raft_consensus.cc:2272] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:03.504056 13403 ts_tablet_manager.cc:1916] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:59:03.519098 12696 consensus_peers.cc:597] T 6f666e1a8d214172bf5b752e9ec374bd P 3ae87963ed5e4d92a896bf4a73af4da9 -> Peer 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): Couldn't send request to peer 8112c0b43d9c4e5389aa3a18f18b58a5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.519479 12696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.195:44901, user_credentials={real_user=slave}} blocked reactor thread for 77261.5us
I20260926 19:59:03.520396 12696 leader_election.cc:304] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
W20260926 19:59:03.522120 13739 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.233.252:44243, user_credentials={real_user=slave}} blocked reactor thread for 67290.9us
I20260926 19:59:03.422554 14109 rpcz_store.cc:276] 0926 19:59:00.354020 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:00.354102 (+ 82us) service_pool.cc:224] Handling call
0926 19:59:03.421904 (+3067802us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:59:00.403812 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0926 19:59:00.427725 (+ 23913us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0926 19:59:00.427756 (+ 31us) alter_schema_op.cc:66] Acquired schema lock
0926 19:59:00.427822 (+ 66us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0926 19:59:00.427953 (+ 131us) alter_schema_op.cc:128] START. Timestamp: 1790452740301413
0926 19:59:00.427969 (+ 16us) op_driver.cc:348] REPLICATION: starting
0926 19:59:00.441696 (+ 13727us) log.cc:844] Serialized 173 byte log entry
0926 19:59:02.929152 (+2487456us) op_driver.cc:464] REPLICATION: finished
0926 19:59:03.038635 (+109483us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0926 19:59:03.062806 (+ 24171us) log.cc:844] Serialized 5 byte log entry
0926 19:59:03.360933 (+298127us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.420930 (+ 59997us) log.cc:844] Serialized 14 byte log entry
0926 19:59:03.421765 (+ 835us) alter_schema_op.cc:73] Released schema lock
0926 19:59:03.421798 (+ 33us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":109107,"prepare.queue_time_us":33891,"prepare.run_cpu_time_us":36526,"prepare.run_wall_time_us":1763278,"replication_time_us":2500883,"thread_start_us":1829477,"threads_started":2,"wal-append.queue_time_us":1687640}]]}
I20260926 19:59:03.493546 14102 raft_consensus.cc:697] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae87963ed5e4d92a896bf4a73af4da9, State: Running, Role: LEADER
I20260926 19:59:03.540387 13092 raft_consensus.cc:385] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:03.540772 13092 raft_consensus.cc:740] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:59:03.540975 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c23ecdde3b04ce784f3f4bf0b2f0441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.541738 13092 consensus_queue.cc:260] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:03.543740 13092 ts_tablet_manager.cc:1434] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent starting tablet: real 0.063s user 0.003s sys 0.004s
I20260926 19:59:03.549897 12857 leader_election.cc:304] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters:
I20260926 19:59:03.551277 14114 raft_consensus.cc:2804] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:03.551585 14114 raft_consensus.cc:493] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:03.551894 14114 raft_consensus.cc:3060] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:03.554085 12732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bac488d42444d07962be0fe9e8ed2e5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:03.555626 12381 connection.cc:573] client connection to 127.11.233.193:46439 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:03.555976 12381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 4 similar messages]
W20260926 19:59:03.557690 12857 connection.cc:573] server connection from 127.11.233.193:39145 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:03.559155 12858 connection.cc:573] client connection to 127.11.233.193:46439 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:03.559595 12858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:59:03.546715 14074 raft_consensus.cc:2804] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:03.560168 12857 leader_election.cc:304] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ae87963ed5e4d92a896bf4a73af4da9, 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:59:03.560822 12857 raft_consensus.cc:2686] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260926 19:59:03.561524 14020 raft_consensus.cc:493] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:03.562063 14020 raft_consensus.cc:515] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:03.564970 14020 leader_election.cc:290] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:03.532460 14102 consensus_queue.cc:237] T 700d05149a934d5e8bec9cc41873ad3e P 3ae87963ed5e4d92a896bf4a73af4da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:03.562523 14074 raft_consensus.cc:493] T 56fd4c88f9304f19883c3e5da8f937b5 P 3ae87963ed5e4d92a896bf4a73af4da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260926 19:59:03.595536 12695 connection.cc:573] client connection to 127.11.233.193:46439 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:03.595992 12695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:59:03.601913 12758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4d86fa6b40b4fc49c4fa67b63792831" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" is_pre_election: true
I20260926 19:59:03.567458 12725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50df02bfe05548dfb7f166adfa39256e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.574196 13012 tablet_bootstrap.cc:492] T 8f59be98ceef4effa15baf487c2d1140 P 3ae87963ed5e4d92a896bf4a73af4da9: Bootstrap starting.
I20260926 19:59:03.607507 12761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fd4c88f9304f19883c3e5da8f937b5" candidate_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae87963ed5e4d92a896bf4a73af4da9"
I20260926 19:59:03.609091 13569 ts_tablet_manager.cc:1916] T 2ad76fc959e24545bdfad19f988fce93 P 3ae87963ed5e4d92a896bf4a73af4da9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:59:03.605705 12857 leader_election.cc:343] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:59:03.611179 14119 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.261s user 0.006s sys 0.017s
W20260926 19:59:03.566423 12694 connection.cc:573] server connection from 127.11.233.193:37639 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:59:03.604305 14114 raft_consensus.cc:515] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } }
I20260926 19:59:03.622478 12727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6d74658dea845a3af17a79f06c444c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.623785 12726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9424c4d822964f40b1feeb2764519d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.623885 14114 leader_election.cc:290] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Requested vote from peers f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439), 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461)
I20260926 19:59:03.634750 13092 tablet_bootstrap.cc:492] T 8f59be98ceef4effa15baf487c2d1140 P 8112c0b43d9c4e5389aa3a18f18b58a5: Bootstrap starting.
I20260926 19:59:03.641659 12199 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu with pid 12645
W20260926 19:59:03.644349 12382 catalog_manager.cc:4845] Async tablet task AlterTable RPC for tablet 13bb4e7eeef145fc85a5c412add99ce7 (table default.Table_019731a69622436792bb7a8512e0cb49 [id=f2cc65dc707940e5a763c0d4b768a072], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 13bb4e7eeef145fc85a5c412add99ce7 consensus state has no leader
W20260926 19:59:03.651784 14141 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 19:59:03.584233 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.11.233.193:46439 (local address 127.0.0.1:37010)
0926 19:59:03.650795 (+ 66562us) negotiation.cc:107] Waiting for socket to connect
0926 19:59:03.651040 (+ 245us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":65937,"threads_started":1}
I20260926 19:59:03.652450 13092 tablet_bootstrap.cc:654] T 8f59be98ceef4effa15baf487c2d1140 P 8112c0b43d9c4e5389aa3a18f18b58a5: Neither blocks nor log segments found. Creating new log.
W20260926 19:59:03.672524 12858 consensus_peers.cc:597] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.672992 12858 consensus_peers.cc:597] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.673352 12381 catalog_manager.cc:4790] TS f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): AlterTable RPC failed for tablet 59d1955b12dd47899b8a226fe0e980ae: Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection reset by peer (error 104)
W20260926 19:59:03.679900 12858 leader_election.cc:336] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
I20260926 19:59:03.680305 12858 leader_election.cc:304] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
W20260926 19:59:03.680909 12858 raft_consensus.cc:2686] T 2ad76fc959e24545bdfad19f988fce93 P 8112c0b43d9c4e5389aa3a18f18b58a5: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260926 19:59:03.682665 12858 consensus_peers.cc:597] T 2411a153e7194406a2bb7b879c08d7dd P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.683022 12858 leader_election.cc:336] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
I20260926 19:59:03.683328 12858 leader_election.cc:304] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:03.690811 14114 raft_consensus.cc:2749] T 1aa82c030ec34ac5a8c448d80cdbae95 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 19:59:03.691768 12858 leader_election.cc:336] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
I20260926 19:59:03.695719 12858 leader_election.cc:304] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:03.696491 14114 raft_consensus.cc:2749] T 0a7d5e580f5645ab95fa03d5f669dc32 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 19:59:03.697391 12858 consensus_peers.cc:597] T 99465590a0e64b35b8a8f07e8777b524 P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.700799 12858 consensus_peers.cc:597] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.701212 12858 leader_election.cc:336] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
W20260926 19:59:03.704435 12858 leader_election.cc:336] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
W20260926 19:59:03.705116 12858 consensus_peers.cc:597] T 62e75eb4b3e8493b9d599c1c4ad4c67f P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Couldn't send request to peer f6e9d6439fd54b28925a68a6145d3dee. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.710031 12858 leader_election.cc:336] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
I20260926 19:59:03.710614 12858 leader_election.cc:304] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:03.711850 14114 raft_consensus.cc:2749] T 55b7715b8fcb4b59917ae14a83f9955d P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 19:59:03.712992 12858 leader_election.cc:336] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
I20260926 19:59:03.716276 12858 leader_election.cc:304] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:03.716985 14114 raft_consensus.cc:2749] T fcad0bbb2bb44338bd56f2547e70ab9c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260926 19:59:03.718089 12858 leader_election.cc:336] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
W20260926 19:59:03.722152 12858 leader_election.cc:336] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
I20260926 19:59:03.738458 12393 catalog_manager.cc:5938] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901) does not have the latest schema for tablet cc9f7a2e5d1c41a8ac793a31696fb311 (table default.Table_24dedc75d9844c41976c575db604e1d7 [id=87b4a0b0de7f410a8843a8c5e7ddee9c]). Expected version 1 got 0
W20260926 19:59:03.739157 12858 leader_election.cc:336] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
I20260926 19:59:03.739537 12858 leader_election.cc:304] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:03.744223 14114 raft_consensus.cc:2749] T c4d86fa6b40b4fc49c4fa67b63792831 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:59:03.745270 12858 leader_election.cc:336] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer f6e9d6439fd54b28925a68a6145d3dee (127.11.233.193:46439): Network error: Client connection negotiation failed: client connection to 127.11.233.193:46439: connect: Connection refused (error 111)
I20260926 19:59:03.750903 12896 tablet_service.cc:1511] Processing CreateTablet for tablet 9387137d90ed4a49a627a330e0d938b0 (DEFAULT_TABLE table=default.Table_380d5450416f4fca9ceebc9625ca9c9f [id=123e385c92254d66b240740d5945bc9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.752008 12896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9387137d90ed4a49a627a330e0d938b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.754513 13092 tablet_bootstrap.cc:492] T 8f59be98ceef4effa15baf487c2d1140 P 8112c0b43d9c4e5389aa3a18f18b58a5: No bootstrap required, opened a new log
I20260926 19:59:03.757720 13092 ts_tablet_manager.cc:1403] T 8f59be98ceef4effa15baf487c2d1140 P 8112c0b43d9c4e5389aa3a18f18b58a5: Time spent bootstrapping tablet: real 0.123s user 0.004s sys 0.008s
I20260926 19:59:03.760059 13092 raft_consensus.cc:359] T 8f59be98ceef4effa15baf487c2d1140 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8112c0b43d9c4e5389aa3a18f18b58a5" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 44901 } } peers { permanent_uuid: "3ae87963ed5e4d92a896bf4a73af4da9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 37461 } } peers { permanent_uuid: "f6e9d6439fd54b28925a68a6145d3dee" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 46439 } }
I20260926 19:59:03.760638 13092 raft_consensus.cc:385] T 8f59be98ceef4effa15baf487c2d1140 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:03.760878 13092 raft_consensus.cc:740] T 8f59be98ceef4effa15baf487c2d1140 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8112c0b43d9c4e5389aa3a18f18b58a5, State: Initialized, Role: FOLLOWER
I20260926 19:59:03.761550 13403 ts_tablet_manager.cc:1929] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:03.763485 13403 log.cc:1199] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/59f7dc92330d400f93f427f3f79ad826
I20260926 19:59:03.767714 12886 tablet_service.cc:1511] Processing CreateTablet for tablet d941729ed3194452bb9763bb7ad20eb0 (DEFAULT_TABLE table=default.Table_272be26b54a546cf80b56c1c7b402235 [id=64a575b967ed447a8fdcdc53cd1db7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.773785 12886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d941729ed3194452bb9763bb7ad20eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.776597 12899 tablet_service.cc:1511] Processing CreateTablet for tablet 50d6cbf3191d4043b79f421a26d5f5ec (DEFAULT_TABLE table=default.Table_4144e2041c67465ea697a20aa6b1800b [id=b4bf3d9f4ea24cd6923d11d90cbe1b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.769315 12888 tablet_service.cc:1511] Processing CreateTablet for tablet 3f5bcf9e7ae849b399457817d6c4f68e (DEFAULT_TABLE table=default.Table_c5597da00ff04e24a0e5f257cd0d3353 [id=e2d91622a45a457b96e662ed8b336b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.777639 12899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50d6cbf3191d4043b79f421a26d5f5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.778340 12888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f5bcf9e7ae849b399457817d6c4f68e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.789143 12887 tablet_service.cc:1511] Processing CreateTablet for tablet f8616b51b994424a91a1370a53f81bde (DEFAULT_TABLE table=default.Table_7d9b396d3f0a41fba07931d6ce8c143d [id=0bdb72f8c6864f0397aa0098e84f6458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.790349 12887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8616b51b994424a91a1370a53f81bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:03.792034 12394 catalog_manager.cc:6391] T 00000000000000000000000000000000 P 8a69f5870f8d461aa5e61f6999bb05c6: default.Table_935b6b15b7c944ab8b9714014acd40e8 [id=0a3fcb2373e24226a1424faaa9083504] alter complete (version 1)
I20260926 19:59:03.798795 12891 tablet_service.cc:1511] Processing CreateTablet for tablet 1e64bfd2fd6d436980b6f2c0bbee7c1b (DEFAULT_TABLE table=default.Table_a73209f0f4544df897c1e856f3692fe8 [id=813b847fffb449f497ef2bc3ff10186a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:03.799965 12891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e64bfd2fd6d436980b6f2c0bbee7c1b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:03.801625 13728 connection.cc:573] server connection from 127.11.233.194:50819 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:59:03.814102 13403 ts_tablet_manager.cc:1950] T 59f7dc92330d400f93f427f3f79ad826 P 8112c0b43d9c4e5389aa3a18f18b58a5: Deleting consensus metadata
W20260926 19:59:03.816033 12860 connection.cc:573] server connection from 127.11.233.194:55801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:03.816062 12380 connection.cc:573] client connection to 127.11.233.194:37461 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:59:03.817165 13403 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33594 (request call id 336) took 2527 ms. Trace:
W20260926 19:59:03.817276 12857 connection.cc:573] client connection to 127.11.233.194:37461 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:59:03.817669 13403 rpcz_store.cc:276] 0926 19:59:01.289882 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:01.291707 (+ 1825us) service_pool.cc:224] Handling call
0926 19:59:03.482737 (+2191030us) ts_tablet_manager.cc:1151] Deleting tablet 59f7dc92330d400f93f427f3f79ad826
0926 19:59:03.498566 (+ 15829us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0926 19:59:03.656003 (+157437us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.757196 (+101193us) tablet_metadata.cc:665] Metadata flushed
0926 19:59:03.817105 (+ 59909us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2190474}
I20260926 19:59:03.818770 13403 tablet_replica.cc:333] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5: stopping tablet replica
I20260926 19:59:03.819540 13403 raft_consensus.cc:2243] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:03.820048 13403 raft_consensus.cc:2272] T 5930402db43740b4b291c6c238ecf52c P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260926 19:59:03.824813 12857 consensus_peers.cc:597] T a5324b1b682f43d4a689b6fafff3df01 P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.194:37461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.825593 12857 leader_election.cc:336] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Network error: Client connection negotiation failed: client connection to 127.11.233.194:37461: connect: Connection refused (error 111)
I20260926 19:59:03.826040 12857 leader_election.cc:304] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:03.826726 14114 raft_consensus.cc:2749] T edb035b50e8f445ba492e67796085123 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 19:59:03.827569 12857 consensus_peers.cc:597] T 89870d2c5b0342c4bdf66474fa8d38fe P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.194:37461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.828032 12857 leader_election.cc:336] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Network error: Client connection negotiation failed: client connection to 127.11.233.194:37461: connect: Connection refused (error 111)
I20260926 19:59:03.828439 12857 leader_election.cc:304] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:03.829125 14114 raft_consensus.cc:2749] T 56fd4c88f9304f19883c3e5da8f937b5 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260926 19:59:03.830128 12857 consensus_peers.cc:597] T 1d0583424ef64d9ea3beeb1c01ceafba P 8112c0b43d9c4e5389aa3a18f18b58a5 -> Peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Couldn't send request to peer 3ae87963ed5e4d92a896bf4a73af4da9. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.194:37461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260926 19:59:03.830559 12857 leader_election.cc:336] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 3ae87963ed5e4d92a896bf4a73af4da9 (127.11.233.194:37461): Network error: Client connection negotiation failed: client connection to 127.11.233.194:37461: connect: Connection refused (error 111)
I20260926 19:59:03.831097 12857 leader_election.cc:304] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8112c0b43d9c4e5389aa3a18f18b58a5; no voters: 3ae87963ed5e4d92a896bf4a73af4da9, f6e9d6439fd54b28925a68a6145d3dee
I20260926 19:59:03.831717 14114 raft_consensus.cc:2749] T 986a28107bd54199bf2cda96978150c3 P 8112c0b43d9c4e5389aa3a18f18b58a5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:59:03.835302 12382 catalog_manager.cc:5063] TS 8112c0b43d9c4e5389aa3a18f18b58a5 (127.11.233.195:44901): tablet 59f7dc92330d400f93f427f3f79ad826 (table default.Table_7a74cde0204a4a97953baf0c9f7b1968 [id=185449ae1c9744eba4df8bd0f9704abb]) successfully deleted
I20260926 19:59:03.837061 12199 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu with pid 12816
I20260926 19:59:04.114594 12199 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu with pid 12208
I20260926 19:59:04.317843 14157 raft_consensus.cc:493] T 00000000000000000000000000000000 P db2490adb6e84090b780f52470e580ab [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a69f5870f8d461aa5e61f6999bb05c6)
I20260926 19:59:04.318717 12199 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu with pid 13130
I20260926 19:59:04.362540 12199 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu with pid 12353
2026-09-26T19:59:04Z chronyd exiting
I20260926 19:59:04.427093 12199 master-stress-test.cc:587] Tables created: 212
I20260926 19:59:04.427284 12199 master-stress-test.cc:588] Tables altered: 99
I20260926 19:59:04.427402 12199 master-stress-test.cc:589] Tables deleted: 79
I20260926 19:59:04.427521 12199 master-stress-test.cc:590] Tablets replaced: 33
I20260926 19:59:04.427635 12199 master-stress-test.cc:591] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (44850 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
2026-09-26T19:59:04Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:59:04Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskT6OA0Z/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskT6OA0Z/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/security.properties
2026-09-26 19:59:11: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskT6OA0Z/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskT6OA0Z/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260926 19:59:20.051584 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.254:32801
--webserver_interface=127.11.233.254
--webserver_port=0
--builtin_ntp_servers=127.11.233.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--hive_metastore_uris=thrift://127.0.0.1:43737
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260926 19:59:20.363349 14439 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:59:20.364017 14439 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:59:20.364494 14439 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:59:20.398849 14439 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 19:59:20.399176 14439 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 19:59:20.399492 14439 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:59:20.399781 14439 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:59:20.400167 14439 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:59:20.400533 14439 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:59:20.439317 14439 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:43737
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.254:32801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.233.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:59:20.440627 14439 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:59:20.442222 14439 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:59:20.452638 14445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:20.454118 14446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:21.696617 14448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:21.698953 14447 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1241 milliseconds
I20260926 19:59:21.699057 14439 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:59:21.700299 14439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:59:21.703454 14439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:59:21.704895 14439 hybrid_clock.cc:648] HybridClock initialized: now 1790452761704851 us; error 59 us; skew 500 ppm
I20260926 19:59:21.705770 14439 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:59:21.712180 14439 webserver.cc:533] Webserver started at http://127.11.233.254:45057/ using document root <none> and password file <none>
I20260926 19:59:21.713133 14439 fs_manager.cc:362] Metadata directory not provided
I20260926 19:59:21.713364 14439 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:59:21.713876 14439 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:59:21.718194 14439 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/instance:
uuid: "26c8ef2c095f418083f33f8f0a734c3a"
format_stamp: "Formatted at 2026-09-26 19:59:21 on dist-test-slave-jcdj"
I20260926 19:59:21.719408 14439 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal/instance:
uuid: "26c8ef2c095f418083f33f8f0a734c3a"
format_stamp: "Formatted at 2026-09-26 19:59:21 on dist-test-slave-jcdj"
I20260926 19:59:21.726845 14439 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260926 19:59:21.732707 14456 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:21.733937 14439 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 19:59:21.734295 14439 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
uuid: "26c8ef2c095f418083f33f8f0a734c3a"
format_stamp: "Formatted at 2026-09-26 19:59:21 on dist-test-slave-jcdj"
I20260926 19:59:21.734648 14439 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:59:21.783748 14439 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:59:21.785251 14439 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:59:21.785744 14439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:59:21.858063 14439 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.254:32801
I20260926 19:59:21.858121 14507 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.254:32801 every 8 connection(s)
I20260926 19:59:21.861168 14439 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
I20260926 19:59:21.862836 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 14439
I20260926 19:59:21.863616 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal/instance
I20260926 19:59:21.866222 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.253:39245
--webserver_interface=127.11.233.253
--webserver_port=0
--builtin_ntp_servers=127.11.233.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--hive_metastore_uris=thrift://127.0.0.1:43737
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 19:59:21.867321 14508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:21.883213 14508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:59:21.913743 14508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } has no permanent_uuid. Determining permanent_uuid...
W20260926 19:59:21.916419 14457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:21.921051 14508 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:21.970530 14508 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } attempt: 1
W20260926 19:59:21.975680 14508 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:22.044126 14508 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } attempt: 2
W20260926 19:59:22.048797 14508 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:22.140236 14508 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } attempt: 3
W20260926 19:59:22.144865 14508 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:22.189231 14509 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:59:22.189895 14509 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:59:22.190351 14509 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:59:22.224507 14509 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 19:59:22.224891 14509 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 19:59:22.225186 14509 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:59:22.225454 14509 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:59:22.225744 14509 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:59:22.226004 14509 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:59:22.277628 14509 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:43737
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.253:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.233.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:59:22.279578 14509 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:59:22.282124 14509 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260926 19:59:22.288280 14508 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } attempt: 4
W20260926 19:59:22.294073 14508 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:22.297315 14518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:22.593573 14508 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } attempt: 5
W20260926 19:59:22.601981 14508 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:23.149576 14508 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } attempt: 6
W20260926 19:59:23.181874 14508 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:23.701045 14517 debug-util.cc:398] Leaking SignalData structure 0x7b08000283c0 after lost signal to thread 14509
W20260926 19:59:24.110656 14509 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.812s user 0.710s sys 1.101s
W20260926 19:59:24.111119 14509 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.813s user 0.710s sys 1.101s
W20260926 19:59:22.298830 14519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:24.113070 14521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:24.116056 14520 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1814 milliseconds
I20260926 19:59:24.116082 14509 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:59:24.117262 14509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:59:24.119683 14509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:59:24.121055 14509 hybrid_clock.cc:648] HybridClock initialized: now 1790452764121026 us; error 37 us; skew 500 ppm
I20260926 19:59:24.121913 14509 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:59:24.128230 14509 webserver.cc:533] Webserver started at http://127.11.233.253:39383/ using document root <none> and password file <none>
I20260926 19:59:24.129200 14509 fs_manager.cc:362] Metadata directory not provided
I20260926 19:59:24.129444 14509 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:59:24.130014 14509 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:59:24.134334 14509 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/instance:
uuid: "fa6dae08245e46c98e01465e84732c92"
format_stamp: "Formatted at 2026-09-26 19:59:24 on dist-test-slave-jcdj"
I20260926 19:59:24.135439 14509 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal/instance:
uuid: "fa6dae08245e46c98e01465e84732c92"
format_stamp: "Formatted at 2026-09-26 19:59:24 on dist-test-slave-jcdj"
I20260926 19:59:24.142868 14509 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260926 19:59:24.148595 14530 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:24.149827 14509 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 19:59:24.150166 14509 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
uuid: "fa6dae08245e46c98e01465e84732c92"
format_stamp: "Formatted at 2026-09-26 19:59:24 on dist-test-slave-jcdj"
I20260926 19:59:24.150509 14509 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:59:24.196933 14509 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:59:24.198479 14509 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:59:24.198930 14509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:59:24.242471 14508 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } attempt: 7
W20260926 19:59:24.251269 14508 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:24.272712 14509 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.253:39245
I20260926 19:59:24.272801 14583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.253:39245 every 8 connection(s)
I20260926 19:59:24.275538 14509 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
I20260926 19:59:24.281358 14584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:24.281338 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 14509
I20260926 19:59:24.281888 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal/instance
I20260926 19:59:24.285475 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.252:37917
--webserver_interface=127.11.233.252
--webserver_port=0
--builtin_ntp_servers=127.11.233.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--hive_metastore_uris=thrift://127.0.0.1:43737
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 19:59:24.297204 14584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:59:24.319097 14584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:59:24.334076 14584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } has no permanent_uuid. Determining permanent_uuid...
W20260926 19:59:24.336480 14534 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 19:59:24.340497 14584 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:37917: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:24.390008 14584 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } attempt: 1
W20260926 19:59:24.394452 14584 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:37917: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:24.462891 14584 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } attempt: 2
W20260926 19:59:24.467263 14584 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:37917: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:24.558688 14584 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } attempt: 3
W20260926 19:59:24.562881 14584 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:37917: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 19:59:24.607774 14585 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:59:24.608404 14585 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:59:24.608853 14585 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:59:24.642057 14585 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 19:59:24.642344 14585 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 19:59:24.642629 14585 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:59:24.642895 14585 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:59:24.643146 14585 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:59:24.643402 14585 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:59:24.681274 14585 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:43737
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.252:37917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.233.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:59:24.682678 14585 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:59:24.684355 14585 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:59:24.694797 14595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:24.706292 14584 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } attempt: 4
W20260926 19:59:24.715857 14584 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:37917: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:25.015421 14584 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } attempt: 5
W20260926 19:59:25.023490 14584 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:37917: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:25.571023 14584 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } attempt: 6
W20260926 19:59:25.599097 14584 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.11.233.252:37917: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 19:59:26.099113 14594 debug-util.cc:398] Leaking SignalData structure 0x7b08000283c0 after lost signal to thread 14585
W20260926 19:59:26.149878 14594 kernel_stack_watchdog.cc:198] Thread 14585 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 400ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260926 19:59:24.696458 14596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:26.150885 14585 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.455s user 0.001s sys 0.001s
W20260926 19:59:26.151279 14585 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.456s user 0.001s sys 0.001s
W20260926 19:59:26.153075 14598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:26.154844 14597 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1453 milliseconds
I20260926 19:59:26.154912 14585 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:59:26.156152 14585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:59:26.158651 14585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:59:26.160029 14585 hybrid_clock.cc:648] HybridClock initialized: now 1790452766159987 us; error 49 us; skew 500 ppm
I20260926 19:59:26.160944 14585 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:59:26.168290 14585 webserver.cc:533] Webserver started at http://127.11.233.252:39091/ using document root <none> and password file <none>
I20260926 19:59:26.170219 14585 fs_manager.cc:362] Metadata directory not provided
I20260926 19:59:26.170572 14585 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:59:26.171176 14585 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:59:26.176362 14585 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/data/instance:
uuid: "eb0f126418d54df489a4ce95a765b8d3"
format_stamp: "Formatted at 2026-09-26 19:59:26 on dist-test-slave-jcdj"
I20260926 19:59:26.177544 14585 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/wal/instance:
uuid: "eb0f126418d54df489a4ce95a765b8d3"
format_stamp: "Formatted at 2026-09-26 19:59:26 on dist-test-slave-jcdj"
I20260926 19:59:26.185030 14585 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260926 19:59:26.190944 14607 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:26.192164 14585 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260926 19:59:26.192528 14585 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/wal
uuid: "eb0f126418d54df489a4ce95a765b8d3"
format_stamp: "Formatted at 2026-09-26 19:59:26 on dist-test-slave-jcdj"
I20260926 19:59:26.192878 14585 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:59:26.255137 14585 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:59:26.256680 14585 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:59:26.257133 14585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:59:26.331532 14585 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.252:37917
I20260926 19:59:26.331617 14658 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.252:37917 every 8 connection(s)
I20260926 19:59:26.334501 14585 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/data/info.pb
I20260926 19:59:26.335095 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 14585
I20260926 19:59:26.335639 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-2/wal/instance
I20260926 19:59:26.341936 14508 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } attempt: 8
I20260926 19:59:26.342700 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:26.367316 14659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:59:26.372232 14508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:59:26.396026 14659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:59:26.402918 14508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Bootstrap starting.
I20260926 19:59:26.407568 14659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:59:26.410447 14508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:26.412962 14508 log.cc:826] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Log is configured to *not* fsync() on all Append() calls
I20260926 19:59:26.418349 14508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: No bootstrap required, opened a new log
I20260926 19:59:26.425149 14659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: Bootstrap starting.
I20260926 19:59:26.431429 14659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:26.433915 14659 log.cc:826] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: Log is configured to *not* fsync() on all Append() calls
I20260926 19:59:26.438899 14659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: No bootstrap required, opened a new log
I20260926 19:59:26.438479 14508 raft_consensus.cc:359] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:26.439131 14508 raft_consensus.cc:385] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:26.439405 14508 raft_consensus.cc:740] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c8ef2c095f418083f33f8f0a734c3a, State: Initialized, Role: FOLLOWER
I20260926 19:59:26.440343 14508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:26.443043 14670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:26.443894 14670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:26.445022 14508 sys_catalog.cc:565] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:59:26.460491 14659 raft_consensus.cc:359] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:26.461539 14659 raft_consensus.cc:385] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:26.461906 14659 raft_consensus.cc:740] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb0f126418d54df489a4ce95a765b8d3, State: Initialized, Role: FOLLOWER
I20260926 19:59:26.462921 14659 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:26.466516 14679 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:26.467629 14679 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:26.468356 14659 sys_catalog.cc:565] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 19:59:26.494237 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.494599 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.512389 14613 tablet.cc:2433] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:59:26.487674 14439 master_runner.cc:187] Error getting master registration for 127.11.233.253:39245: OK, instance_id { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" instance_seqno: 1790452764251842 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260926 19:59:26.521423 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.521883 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.521948 14439 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 19:59:26.537618 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.546952 14462 tablet.cc:2433] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:59:26.550925 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.569736 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.570047 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.585667 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.586208 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.600432 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.600875 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.615151 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.615592 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.629689 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.630118 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.645386 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.649942 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.660008 14584 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } attempt: 7
W20260926 19:59:26.684028 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.684610 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.692926 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.693327 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.699172 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.699509 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.707880 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.708271 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.714900 14584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Bootstrap starting.
I20260926 19:59:26.680644 14585 master_runner.cc:187] Error getting master registration for 127.11.233.253:39245: OK, instance_id { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" instance_seqno: 1790452764251842 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:59:26.716706 14585 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 19:59:26.721235 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.722292 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.722553 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.722922 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.723292 14584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:26.725907 14584 log.cc:826] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Log is configured to *not* fsync() on all Append() calls
W20260926 19:59:26.736675 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.737033 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.737159 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.737473 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.738118 14584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: No bootstrap required, opened a new log
W20260926 19:59:26.751493 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.751662 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.751976 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.751919 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.766355 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.766353 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.766675 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.766764 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.768018 14584 raft_consensus.cc:359] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:26.768752 14584 raft_consensus.cc:385] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:26.769086 14584 raft_consensus.cc:740] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa6dae08245e46c98e01465e84732c92, State: Initialized, Role: FOLLOWER
I20260926 19:59:26.770011 14584 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:26.773089 14707 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:26.774003 14707 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:26.774897 14584 sys_catalog.cc:565] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 19:59:26.799085 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.800071 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.800535 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.800956 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.816211 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.836313 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.839241 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.839834 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.860615 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.868384 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 19:59:26.877889 14670 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:26.882365 14670 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:26.887804 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:26.888643 14670 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:26.888784 14557 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c8ef2c095f418083f33f8f0a734c3a in term 0.
W20260926 19:59:26.889205 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.889582 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.890261 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:26.891181 14634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c8ef2c095f418083f33f8f0a734c3a in term 0.
I20260926 19:59:26.892840 14460 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3; no voters:
I20260926 19:59:26.893957 14670 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:26.894832 14670 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:26.895174 14670 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:59:26.920753 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.921186 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.921612 14670 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:26.928646 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92"
I20260926 19:59:26.929265 14557 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:26.936328 14557 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c8ef2c095f418083f33f8f0a734c3a in term 1.
I20260926 19:59:26.937817 14457 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92; no voters:
W20260926 19:59:26.940357 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.941030 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.941378 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3"
I20260926 19:59:26.942071 14634 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:59:26.947984 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.949198 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:26.950953 14634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c8ef2c095f418083f33f8f0a734c3a in term 1.
I20260926 19:59:26.952569 14670 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 1 election: Requested vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:26.953147 14670 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:26.953629 14670 raft_consensus.cc:697] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 1 LEADER]: Becoming Leader. State: Replica: 26c8ef2c095f418083f33f8f0a734c3a, State: Running, Role: LEADER
I20260926 19:59:26.955554 14670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
W20260926 19:59:26.956040 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.956466 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:26.958122 14536 tablet.cc:2433] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:59:26.974987 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.991931 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:26.997265 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.000761 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.004637 14724 catalog_manager.cc:818] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 19:59:26.994436 14718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c8ef2c095f418083f33f8f0a734c3a. Latest consensus state: current_term: 1 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:27.005887 14718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: LEADER
W20260926 19:59:27.022811 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.023234 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.033290 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.037108 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.037534 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.037925 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.047015 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.047530 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.055800 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.056247 14686 catalog_manager.cc:1607] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.056584 14686 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.056951 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.057636 14725 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
W20260926 19:59:27.071288 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 19:59:27.072938 14725 catalog_manager.cc:1533] Initializing Kudu cluster ID...
W20260926 19:59:27.073292 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.073427 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.078846 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.087604 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.088217 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.094952 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.096182 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.102594 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.102986 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.117175 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.117620 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.122948 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.123324 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.126497 14634 raft_consensus.cc:1275] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 1 FOLLOWER]: Refusing update from remote peer 26c8ef2c095f418083f33f8f0a734c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:59:27.128412 14718 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:59:27.132848 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.133157 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.140269 14557 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 1 FOLLOWER]: Refusing update from remote peer 26c8ef2c095f418083f33f8f0a734c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:59:27.142120 14670 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:59:27.148852 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.149226 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.178363 14707 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c8ef2c095f418083f33f8f0a734c3a. Latest consensus state: current_term: 1 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:27.179098 14707 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 19:59:27.187319 14724 catalog_manager.cc:1607] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.188449 14724 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.198614 14727 mvcc.cc:204] Tried to move back new op lower bound from 7333694534123073536 to 7333694533460353024. Current Snapshot: MvccSnapshot[applied={T|T < 7333694534123073536}]
I20260926 19:59:27.197630 14729 mvcc.cc:204] Tried to move back new op lower bound from 7333694534123073536 to 7333694533460353024. Current Snapshot: MvccSnapshot[applied={T|T < 7333694534123073536}]
I20260926 19:59:27.206523 14707 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:27.207346 14707 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:27.214994 14718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:27.215703 14718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:27.223832 14724 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:27.225685 14725 catalog_manager.cc:1396] Generated new cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:27.226002 14725 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:59:27.227134 14724 catalog_manager.cc:1601] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: success
W20260926 19:59:27.231375 14724 catalog_manager.cc:1625] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: acquiring CA information for follower catalog manager: Not found: root CA entry not found
I20260926 19:59:27.236119 14670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:27.236832 14670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: LEADER
W20260926 19:59:27.238739 14724 catalog_manager.cc:896] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.240826 14728 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c8ef2c095f418083f33f8f0a734c3a. Latest consensus state: current_term: 1 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:27.241541 14728 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 19:59:27.246251 14703 catalog_manager.cc:1607] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:59:27.246595 14703 catalog_manager.cc:896] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.253865 14724 catalog_manager.cc:1625] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260926 19:59:27.257968 14724 catalog_manager.cc:896] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.269621 14728 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:27.270318 14728 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:27.271339 14703 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:27.271651 14703 catalog_manager.cc:1601] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: loading cluster ID for follower catalog manager: success
W20260926 19:59:27.275956 14703 catalog_manager.cc:1625] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260926 19:59:27.277866 14724 catalog_manager.cc:1625] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260926 19:59:27.278573 14724 catalog_manager.cc:896] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.283825 14703 catalog_manager.cc:896] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.284859 14725 catalog_manager.cc:1419] Generated new certificate authority record
I20260926 19:59:27.293708 14725 catalog_manager.cc:1553] Loading token signing keys...
W20260926 19:59:27.298983 14703 catalog_manager.cc:1625] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260926 19:59:27.299310 14703 catalog_manager.cc:896] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.305907 14724 catalog_manager.cc:1623] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: acquiring CA information for follower catalog manager: success
W20260926 19:59:27.314360 14703 catalog_manager.cc:1625] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260926 19:59:27.314666 14703 catalog_manager.cc:896] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
W20260926 19:59:27.320638 14724 catalog_manager.cc:896] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.335765 14703 catalog_manager.cc:1623] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: acquiring CA information for follower catalog manager: success
I20260926 19:59:27.349915 14725 catalog_manager.cc:6219] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Generated new TSK 0
I20260926 19:59:27.351188 14725 catalog_manager.cc:1563] Initializing in-progress tserver states...
W20260926 19:59:27.355007 14724 catalog_manager.cc:896] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.357973 14725 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
W20260926 19:59:27.358253 14703 catalog_manager.cc:896] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260926 19:59:27.362589 14725 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260926 19:59:27.372056 14724 catalog_manager.cc:1651] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:59:27.374526 14703 catalog_manager.cc:1651] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:59:27.414243 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.233.193:0
--local_ip_for_outbound_sockets=127.11.233.193
--webserver_interface=127.11.233.193
--webserver_port=0
--tserver_master_addrs=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--builtin_ntp_servers=127.11.233.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 19:59:27.850133 14739 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:27.850641 14739 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:27.857048 14739 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:27.858810 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:27.860316 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:27.856353 14738 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:27.862157 14738 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:27.864151 14634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa6dae08245e46c98e01465e84732c92 in term 1.
I20260926 19:59:27.867182 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a
I20260926 19:59:27.867961 14739 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 19:59:27.868286 14739 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:27.868640 14739 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:59:27.876055 14739 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:27.876423 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:27.879302 14738 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:27.879771 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:27.880244 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a"
I20260926 19:59:27.880565 14557 raft_consensus.cc:2393] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb0f126418d54df489a4ce95a765b8d3 in current term 2: Already voted for candidate fa6dae08245e46c98e01465e84732c92 in this term.
I20260926 19:59:27.882304 14608 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:27.883145 14738 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:59:27.887422 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3"
I20260926 19:59:27.888831 14739 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 2 election: Requested vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:27.890136 14738 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 19:59:27.902784 14634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa6dae08245e46c98e01465e84732c92 in term 2.
I20260926 19:59:27.905680 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a
I20260926 19:59:27.906503 14739 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:59:27.916623 14739 raft_consensus.cc:697] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 2 LEADER]: Becoming Leader. State: Replica: fa6dae08245e46c98e01465e84732c92, State: Running, Role: LEADER
I20260926 19:59:27.918471 14739 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:27.924489 14740 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa6dae08245e46c98e01465e84732c92. Latest consensus state: current_term: 2 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:27.925220 14740 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:27.945112 14634 raft_consensus.cc:1240] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Rejecting Update request from peer 26c8ef2c095f418083f33f8f0a734c3a for earlier term 1. Current term is 2. Ops: []
I20260926 19:59:27.961153 14557 raft_consensus.cc:1240] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 2 LEADER]: Rejecting Update request from peer 26c8ef2c095f418083f33f8f0a734c3a for earlier term 1. Current term is 2. Ops: []
I20260926 19:59:27.960789 14747 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260926 19:59:27.971143 14750 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 19:59:27.971932 14750 consensus_queue.cc:1243] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Peer fa6dae08245e46c98e01465e84732c92 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260926 19:59:27.974581 14743 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 19:59:27.978654 14749 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 1 LEADER]: Stepping down as leader of term 1
I20260926 19:59:27.979115 14749 raft_consensus.cc:740] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 26c8ef2c095f418083f33f8f0a734c3a, State: Running, Role: LEADER
I20260926 19:59:27.979990 14749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:27.981102 14749 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:59:27.996933 14743 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 19:59:28.001379 14743 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:28.001696 14743 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:59:28.004283 14686 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:59:28.016528 14743 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:59:28.021185 14743 catalog_manager.cc:6230] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Loaded TSK: 0
I20260926 19:59:28.027781 14743 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:59:28.037441 14743 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 19:59:28.041857 14743 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
W20260926 19:59:28.074249 14737 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:59:28.074944 14737 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:59:28.076015 14737 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:59:28.139246 14737 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:59:28.140805 14737 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.233.193
I20260926 19:59:28.209646 14737 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.233.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.233.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.233.193
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:59:28.211380 14737 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:59:28.213541 14737 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:59:28.248373 14753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:28.436637 14634 raft_consensus.cc:1275] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Refusing update from remote peer fa6dae08245e46c98e01465e84732c92: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260926 19:59:28.438714 14740 consensus_queue.cc:1048] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260926 19:59:28.457376 14749 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:28.457907 14749 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:28.458118 14758 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa6dae08245e46c98e01465e84732c92. Latest consensus state: current_term: 2 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:28.459016 14758 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:28.459995 14749 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:28.461740 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:28.462406 14633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:28.465014 14460 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a; no voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:28.466229 14749 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260926 19:59:28.484728 14758 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:28.485443 14758 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:28.496170 14483 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Refusing update from remote peer fa6dae08245e46c98e01465e84732c92: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260926 19:59:28.502349 14740 consensus_queue.cc:1048] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260926 19:59:28.507403 14760 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:28.513463 14760 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:28.547580 14766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:28.547950 14749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa6dae08245e46c98e01465e84732c92. Latest consensus state: current_term: 2 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:28.548458 14766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:28.548628 14749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:28.980940 14758 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fa6dae08245e46c98e01465e84732c92)
I20260926 19:59:28.981549 14758 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:28.984010 14758 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:28.985301 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:28.987160 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:28.989037 14611 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:28.990124 14758 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260926 19:59:29.020181 14749 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader fa6dae08245e46c98e01465e84732c92)
I20260926 19:59:29.020818 14749 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:29.023273 14749 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:29.024538 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:29.025195 14634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c8ef2c095f418083f33f8f0a734c3a in term 2.
I20260926 19:59:29.026654 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:29.028959 14460 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3; no voters: fa6dae08245e46c98e01465e84732c92
I20260926 19:59:29.030185 14749 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 19:59:29.030535 14749 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Starting leader election (detected failure of leader fa6dae08245e46c98e01465e84732c92)
I20260926 19:59:29.030923 14749 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 2 FOLLOWER]: Advancing to term 3
I20260926 19:59:29.038496 14749 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:29.044966 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92"
I20260926 19:59:29.048580 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3"
I20260926 19:59:29.049160 14634 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 2 FOLLOWER]: Advancing to term 3
I20260926 19:59:29.058023 14634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c8ef2c095f418083f33f8f0a734c3a in term 3.
I20260926 19:59:29.060513 14460 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3; no voters: fa6dae08245e46c98e01465e84732c92
I20260926 19:59:29.067744 14767 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 FOLLOWER]: Leader election won for term 3
I20260926 19:59:29.069331 14749 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 3 election: Requested vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:29.081094 14767 raft_consensus.cc:697] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 LEADER]: Becoming Leader. State: Replica: 26c8ef2c095f418083f33f8f0a734c3a, State: Running, Role: LEADER
I20260926 19:59:29.083849 14767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:29.099401 14634 raft_consensus.cc:1240] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Rejecting Update request from peer fa6dae08245e46c98e01465e84732c92 for earlier term 2. Current term is 3. Ops: []
I20260926 19:59:29.100860 14749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c8ef2c095f418083f33f8f0a734c3a. Latest consensus state: current_term: 3 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:29.101670 14749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:29.114540 14769 consensus_queue.cc:1059] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 19:59:29.129572 14773 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:29.136492 14770 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 19:59:29.130905 14771 raft_consensus.cc:3055] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 2 LEADER]: Stepping down as leader of term 2
I20260926 19:59:29.139081 14771 raft_consensus.cc:740] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fa6dae08245e46c98e01465e84732c92, State: Running, Role: LEADER
I20260926 19:59:29.140628 14771 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:29.142025 14771 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 2 FOLLOWER]: Advancing to term 3
I20260926 19:59:29.152199 14773 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:29.162186 14483 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 LEADER]: Rejecting Update request from peer fa6dae08245e46c98e01465e84732c92 for earlier term 2. Current term is 3. Ops: []
I20260926 19:59:29.164539 14770 catalog_manager.cc:1533] Initializing Kudu cluster ID...
W20260926 19:59:29.175235 14774 catalog_manager.cc:1458] Failed waiting for node to catch up after master election: Illegal state: Node fa6dae08245e46c98e01465e84732c92 not leader. Raft Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:29.176033 14770 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:29.176362 14770 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:59:29.187520 14770 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:59:29.194167 14770 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Loaded TSK: 0
I20260926 19:59:29.201538 14770 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:59:29.211475 14770 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 19:59:29.220979 14770 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260926 19:59:29.615309 14634 raft_consensus.cc:1275] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Refusing update from remote peer 26c8ef2c095f418083f33f8f0a734c3a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260926 19:59:29.617091 14749 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 19:59:29.658879 14775 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c8ef2c095f418083f33f8f0a734c3a. Latest consensus state: current_term: 3 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:29.661953 14775 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:29.672788 14775 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:29.674233 14775 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:29.679239 14557 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Refusing update from remote peer 26c8ef2c095f418083f33f8f0a734c3a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260926 19:59:29.680953 14749 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 19:59:29.703168 14777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:29.706625 14777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:29.735337 14782 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:29.735498 14781 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c8ef2c095f418083f33f8f0a734c3a. Latest consensus state: current_term: 3 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:29.736157 14781 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:29.735929 14782 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:29.740322 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:29.740332 14785 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:29.740980 14785 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:29.744484 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:29.746727 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:29.748121 14785 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:29.749079 14782 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:30.168686 14775 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:30.169327 14775 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:30.174186 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:30.174662 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:30.175868 14775 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:30.177294 14608 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:30.178898 14775 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:30.186883 14777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:30.187602 14777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: LEADER
W20260926 19:59:29.637023 14751 debug-util.cc:398] Leaking SignalData structure 0x7b0800037180 after lost signal to thread 14737
W20260926 19:59:28.250070 14752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:30.347807 14751 kernel_stack_watchdog.cc:198] Thread 14737 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 382ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260926 19:59:30.370612 14754 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection timed out after 2118 milliseconds
W20260926 19:59:30.371568 14737 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 2.119s user 0.000s sys 0.007s
W20260926 19:59:30.372027 14737 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 2.119s user 0.001s sys 0.007s
W20260926 19:59:30.372463 14756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:30.372583 14737 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:59:30.374393 14737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:59:30.411069 14737 hybrid_clock.cc:648] HybridClock initialized: now 1790452770411034 us; error 69 us; skew 500 ppm
I20260926 19:59:30.412289 14737 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:59:30.435609 14737 webserver.cc:533] Webserver started at http://127.11.233.193:45797/ using document root <none> and password file <none>
I20260926 19:59:30.437081 14737 fs_manager.cc:362] Metadata directory not provided
I20260926 19:59:30.437454 14737 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:59:30.438180 14737 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:59:30.444958 14737 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/data/instance:
uuid: "56196546dad349b9bd344c4e0b6a6874"
format_stamp: "Formatted at 2026-09-26 19:59:30 on dist-test-slave-jcdj"
I20260926 19:59:30.446643 14737 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/instance:
uuid: "56196546dad349b9bd344c4e0b6a6874"
format_stamp: "Formatted at 2026-09-26 19:59:30 on dist-test-slave-jcdj"
I20260926 19:59:30.463493 14737 fs_manager.cc:696] Time spent creating directory manager: real 0.016s user 0.009s sys 0.004s
I20260926 19:59:30.477964 14791 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:30.479456 14737 fs_manager.cc:730] Time spent opening block manager: real 0.011s user 0.006s sys 0.004s
I20260926 19:59:30.479956 14737 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal
uuid: "56196546dad349b9bd344c4e0b6a6874"
format_stamp: "Formatted at 2026-09-26 19:59:30 on dist-test-slave-jcdj"
I20260926 19:59:30.480546 14737 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:59:30.573881 14737 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:59:30.576047 14737 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:59:30.576789 14737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:59:30.586572 14737 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 19:59:30.599720 14737 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:59:30.600157 14737 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 19:59:30.600559 14737 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 19:59:30.600860 14737 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:31.011063 14737 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.193:40755
I20260926 19:59:31.011188 14904 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.193:40755 every 8 connection(s)
I20260926 19:59:31.014959 14737 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/data/info.pb
I20260926 19:59:31.016623 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 14737
I20260926 19:59:31.017426 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/instance
I20260926 19:59:31.034824 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.233.194:0
--local_ip_for_outbound_sockets=127.11.233.194
--webserver_interface=127.11.233.194
--webserver_port=0
--tserver_master_addrs=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--builtin_ntp_servers=127.11.233.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 19:59:31.105067 14905 heartbeater.cc:344] Connected to a master server at 127.11.233.253:39245
I20260926 19:59:31.105798 14905 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:31.107563 14905 heartbeater.cc:507] Master 127.11.233.253:39245 requested a full tablet report, sending...
I20260926 19:59:31.109627 14906 heartbeater.cc:344] Connected to a master server at 127.11.233.252:37917
I20260926 19:59:31.109994 14906 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:31.110900 14906 heartbeater.cc:507] Master 127.11.233.252:37917 requested a full tablet report, sending...
I20260926 19:59:31.112126 14547 ts_manager.cc:194] Registered new tserver with Master: 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755)
I20260926 19:59:31.114954 14624 ts_manager.cc:194] Registered new tserver with Master: 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755)
I20260926 19:59:31.124804 14909 heartbeater.cc:344] Connected to a master server at 127.11.233.254:32801
I20260926 19:59:31.125229 14909 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:31.126169 14909 heartbeater.cc:507] Master 127.11.233.254:32801 requested a full tablet report, sending...
I20260926 19:59:31.129282 14473 ts_manager.cc:194] Registered new tserver with Master: 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755)
I20260926 19:59:31.132560 14473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.11.233.193:48457
W20260926 19:59:31.521471 14912 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:59:31.522173 14912 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:59:31.522779 14912 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:59:31.561470 14912 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:59:31.562541 14912 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.233.194
I20260926 19:59:31.566545 14920 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:31.566958 14920 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:31.568754 14920 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:31.569849 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:31.569995 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:31.571671 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:31.572572 14920 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:31.590720 14921 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:31.591171 14921 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:31.592767 14921 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:31.593832 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:31.593928 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:31.595422 14608 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:31.596078 14921 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:31.606721 14912 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.233.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.233.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.233.194
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:59:31.608114 14912 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:59:31.609951 14912 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:59:31.623351 14923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:31.636828 14909 heartbeater.cc:499] Master 127.11.233.254:32801 was elected leader, sending a full tablet report...
I20260926 19:59:32.176496 14929 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:32.176924 14929 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:32.178560 14929 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:32.179371 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:32.180277 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:32.182250 14608 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:32.184890 14929 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:32.558990 14931 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:32.559653 14931 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:32.561269 14931 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:32.562542 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:32.563311 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:32.565099 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:32.565865 14931 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260926 19:59:31.625847 14924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:31.632200 14926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:31.631523 14912 server_base.cc:1061] running on GCE node
I20260926 19:59:32.850395 14912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:59:32.852902 14912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:59:32.854256 14912 hybrid_clock.cc:648] HybridClock initialized: now 1790452772854230 us; error 55 us; skew 500 ppm
I20260926 19:59:32.855104 14912 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:59:32.861464 14912 webserver.cc:533] Webserver started at http://127.11.233.194:40777/ using document root <none> and password file <none>
I20260926 19:59:32.862753 14912 fs_manager.cc:362] Metadata directory not provided
I20260926 19:59:32.863032 14912 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:59:32.863595 14912 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:59:32.868031 14912 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/data/instance:
uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
format_stamp: "Formatted at 2026-09-26 19:59:32 on dist-test-slave-jcdj"
I20260926 19:59:32.869455 14912 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/instance:
uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
format_stamp: "Formatted at 2026-09-26 19:59:32 on dist-test-slave-jcdj"
I20260926 19:59:32.878049 14912 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260926 19:59:32.884960 14936 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:32.886270 14912 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260926 19:59:32.886569 14912 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal
uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
format_stamp: "Formatted at 2026-09-26 19:59:32 on dist-test-slave-jcdj"
I20260926 19:59:32.886880 14912 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:59:32.956379 14912 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:59:32.957973 14912 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:59:32.958455 14912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:59:32.961357 14912 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 19:59:32.965759 14912 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:59:32.965960 14912 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:32.966254 14912 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 19:59:32.966415 14912 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:33.111040 15048 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:33.112128 14912 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.194:34597
I20260926 19:59:33.111681 15048 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:33.112283 15049 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.194:34597 every 8 connection(s)
I20260926 19:59:33.113586 15048 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:33.114452 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:33.114656 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:33.115572 14912 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/data/info.pb
I20260926 19:59:33.116145 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:33.116752 15048 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:33.117581 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 14912
I20260926 19:59:33.118042 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/instance
I20260926 19:59:33.132766 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.233.195:0
--local_ip_for_outbound_sockets=127.11.233.195
--webserver_interface=127.11.233.195
--webserver_port=0
--tserver_master_addrs=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--builtin_ntp_servers=127.11.233.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260926 19:59:33.177331 15051 heartbeater.cc:344] Connected to a master server at 127.11.233.252:37917
I20260926 19:59:33.177690 15050 heartbeater.cc:344] Connected to a master server at 127.11.233.253:39245
I20260926 19:59:33.177901 15051 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:33.178081 15050 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:33.177682 15052 heartbeater.cc:344] Connected to a master server at 127.11.233.254:32801
I20260926 19:59:33.178653 15052 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:33.179386 15050 heartbeater.cc:507] Master 127.11.233.253:39245 requested a full tablet report, sending...
I20260926 19:59:33.179349 15051 heartbeater.cc:507] Master 127.11.233.252:37917 requested a full tablet report, sending...
I20260926 19:59:33.179701 15052 heartbeater.cc:507] Master 127.11.233.254:32801 requested a full tablet report, sending...
I20260926 19:59:33.182638 14547 ts_manager.cc:194] Registered new tserver with Master: 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:33.182631 14473 ts_manager.cc:194] Registered new tserver with Master: 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:33.182731 14624 ts_manager.cc:194] Registered new tserver with Master: 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:33.184381 14473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.11.233.194:60705
I20260926 19:59:33.440244 15061 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:33.440675 15061 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:33.442272 15061 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:33.443255 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:33.443467 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:33.444941 14608 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:33.445600 15061 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260926 19:59:33.489544 15060 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:59:33.490139 15060 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:59:33.490651 15060 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:59:33.525583 15060 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:59:33.526446 15060 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.233.195
I20260926 19:59:33.566097 15060 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.233.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.233.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.233.195
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:59:33.567365 15060 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:59:33.568912 15060 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:59:33.583110 15067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:33.688573 15052 heartbeater.cc:499] Master 127.11.233.254:32801 was elected leader, sending a full tablet report...
I20260926 19:59:34.007589 15073 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:34.008227 15073 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:34.011997 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:34.015622 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:34.017859 14608 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:34.023629 15073 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:34.025735 15073 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:34.617790 15076 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:34.618597 15076 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:34.622391 15076 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:34.623637 14483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:34.681545 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:34.684463 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:34.685616 15076 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260926 19:59:34.986228 15066 debug-util.cc:398] Leaking SignalData structure 0x7b0800037180 after lost signal to thread 15060
W20260926 19:59:33.583763 15068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:35.362757 15069 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1777 milliseconds
W20260926 19:59:35.361248 15060 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.778s user 0.597s sys 1.099s
W20260926 19:59:35.363538 15070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:35.364216 15060 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.782s user 0.597s sys 1.099s
I20260926 19:59:35.364542 15060 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:59:35.369035 15060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:59:35.375842 15060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:59:35.377422 15060 hybrid_clock.cc:648] HybridClock initialized: now 1790452775377282 us; error 148 us; skew 500 ppm
I20260926 19:59:35.378436 15060 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:59:35.385470 15060 webserver.cc:533] Webserver started at http://127.11.233.195:46581/ using document root <none> and password file <none>
I20260926 19:59:35.386494 15060 fs_manager.cc:362] Metadata directory not provided
I20260926 19:59:35.386732 15060 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:59:35.387209 15060 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:59:35.391957 15060 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/data/instance:
uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
format_stamp: "Formatted at 2026-09-26 19:59:35 on dist-test-slave-jcdj"
I20260926 19:59:35.393118 15060 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/instance:
uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
format_stamp: "Formatted at 2026-09-26 19:59:35 on dist-test-slave-jcdj"
I20260926 19:59:35.401008 15060 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260926 19:59:35.407413 15081 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:35.408842 15060 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 19:59:35.409214 15060 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal
uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
format_stamp: "Formatted at 2026-09-26 19:59:35 on dist-test-slave-jcdj"
I20260926 19:59:35.409587 15060 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:59:35.467343 15060 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:59:35.468966 15060 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:59:35.469470 15060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:59:35.472257 15060 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260926 19:59:35.476962 15060 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:59:35.477280 15060 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:35.477623 15060 ts_tablet_manager.cc:616] Registered 0 tablets
I20260926 19:59:35.477852 15060 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:35.622072 15060 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.195:46633
I20260926 19:59:35.622189 15193 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.195:46633 every 8 connection(s)
I20260926 19:59:35.625065 15060 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/data/info.pb
I20260926 19:59:35.627096 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 15060
I20260926 19:59:35.627645 12199 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/instance
I20260926 19:59:35.667780 15194 heartbeater.cc:344] Connected to a master server at 127.11.233.253:39245
I20260926 19:59:35.668296 15194 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:35.669696 15194 heartbeater.cc:507] Master 127.11.233.253:39245 requested a full tablet report, sending...
I20260926 19:59:35.670500 15195 heartbeater.cc:344] Connected to a master server at 127.11.233.252:37917
I20260926 19:59:35.670859 15195 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:35.671708 15195 heartbeater.cc:507] Master 127.11.233.252:37917 requested a full tablet report, sending...
I20260926 19:59:35.672940 14546 ts_manager.cc:194] Registered new tserver with Master: e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:35.675555 14623 ts_manager.cc:194] Registered new tserver with Master: e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:35.678397 15196 heartbeater.cc:344] Connected to a master server at 127.11.233.254:32801
I20260926 19:59:35.678707 15196 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:35.679433 15196 heartbeater.cc:507] Master 127.11.233.254:32801 requested a full tablet report, sending...
I20260926 19:59:35.681370 14473 ts_manager.cc:194] Registered new tserver with Master: e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:35.682713 12199 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260926 19:59:35.682940 14473 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.11.233.195:42021
I20260926 19:59:35.732029 14473 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39068:
name: "default.Table_131b6490764e47dc8a7a589300bbf272"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:35.733605 14471 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39068:
name: "default.Table_7a1c4598445a4ac5845670a68237cb86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:35.738902 14472 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39068:
name: "default.Table_e144b6535e8448a3b04ecd97e88b06a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:35.739065 14470 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39068:
name: "default.Table_214fa130d427469eaf37415c0767c7e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:35.747583 14471 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7a1c4598445a4ac5845670a68237cb86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:35.747611 14473 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_131b6490764e47dc8a7a589300bbf272 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:35.761806 14470 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_214fa130d427469eaf37415c0767c7e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:35.761977 14472 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e144b6535e8448a3b04ecd97e88b06a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:35.787065 12199 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu with pid 14439
W20260926 19:59:35.831779 15207 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.233.254:32801)
I20260926 19:59:35.937472 15229 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:35.938340 15229 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:35.951434 15229 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:35.951529 14557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
W20260926 19:59:35.959404 14611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:35.971365 14611 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 19:59:35.972008 14611 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:35.972820 15229 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:35.993405 15232 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:35.993947 15232 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:35.996271 15232 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:36.025486 14534 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260926 19:59:36.026115 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:36.026636 14634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa6dae08245e46c98e01465e84732c92 in term 3.
I20260926 19:59:36.030671 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92; no voters:
I20260926 19:59:36.031451 15232 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260926 19:59:36.031751 15232 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:36.032109 15232 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 3 FOLLOWER]: Advancing to term 4
I20260926 19:59:36.041639 15232 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
W20260926 19:59:36.042378 14534 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 19:59:36.045948 15232 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 election: Requested vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:36.047474 14634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6dae08245e46c98e01465e84732c92" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3"
I20260926 19:59:36.048033 14634 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 3 FOLLOWER]: Advancing to term 4
W20260926 19:59:36.050711 14534 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 19:59:36.056305 14634 raft_consensus.cc:2468] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa6dae08245e46c98e01465e84732c92 in term 4.
I20260926 19:59:36.057611 14534 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a
I20260926 19:59:36.058393 15232 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 4 FOLLOWER]: Leader election won for term 4
I20260926 19:59:36.072713 15232 raft_consensus.cc:697] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 4 LEADER]: Becoming Leader. State: Replica: fa6dae08245e46c98e01465e84732c92, State: Running, Role: LEADER
I20260926 19:59:36.073693 15232 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:36.082969 15235 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa6dae08245e46c98e01465e84732c92. Latest consensus state: current_term: 4 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:36.083840 15235 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:36.094782 15237 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 19:59:36.103814 15237 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 19:59:36.109714 15237 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:36.109999 15237 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:59:36.114876 15237 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:59:36.118988 15237 catalog_manager.cc:6230] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Loaded TSK: 0
I20260926 19:59:36.120078 15237 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:59:36.123993 15237 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 19:59:36.128024 15237 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260926 19:59:36.140008 14545 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53020:
name: "default.Table_7a1c4598445a4ac5845670a68237cb86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:36.191538 15196 heartbeater.cc:499] Master 127.11.233.254:32801 was elected leader, sending a full tablet report...
I20260926 19:59:36.141681 14547 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53020:
name: "default.Table_214fa130d427469eaf37415c0767c7e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:36.143431 14544 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53020:
name: "default.Table_131b6490764e47dc8a7a589300bbf272"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:36.139671 14546 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53020:
name: "default.Table_e144b6535e8448a3b04ecd97e88b06a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:59:36.201370 14909 heartbeater.cc:646] Failed to heartbeat to 127.11.233.254:32801 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:36.204671 14545 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7a1c4598445a4ac5845670a68237cb86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:36.206557 14547 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_214fa130d427469eaf37415c0767c7e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:36.208098 14544 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_131b6490764e47dc8a7a589300bbf272 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260926 19:59:36.210914 15196 heartbeater.cc:646] Failed to heartbeat to 127.11.233.254:32801 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:36.249898 15052 heartbeater.cc:646] Failed to heartbeat to 127.11.233.254:32801 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 19:59:36.447624 14547 catalog_manager.cc:2213] created HMS catalog entry for table [id=a07796662b5d4b0b8b019aa4e111b875]
W20260926 19:59:36.456276 14546 catalog_manager.cc:7226] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e144b6535e8448a3b04ecd97e88b06a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:59:36.471302 14634 raft_consensus.cc:1275] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Refusing update from remote peer fa6dae08245e46c98e01465e84732c92: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20260926 19:59:36.473469 15235 consensus_queue.cc:1048] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 19:59:36.484711 14534 consensus_peers.cc:597] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 -> Peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Couldn't send request to peer 26c8ef2c095f418083f33f8f0a734c3a. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 19:59:36.518391 15242 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa6dae08245e46c98e01465e84732c92. Latest consensus state: current_term: 4 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:36.519141 15242 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:36.551039 15232 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:36.551782 15232 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:36.555146 15242 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "fa6dae08245e46c98e01465e84732c92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:36.555823 15242 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:36.566036 14547 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53020:
table { table_name: "default.Table_214fa130d427469eaf37415c0767c7e3" } new_table_name: "default.Table_110ad590096942f0a032e632ccedc05f" modify_external_catalogs: true
I20260926 19:59:36.637374 14984 tablet_service.cc:1511] Processing CreateTablet for tablet 8059d2e14ad344088a88501e2ba66ebc (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.639433 14984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8059d2e14ad344088a88501e2ba66ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.655797 14840 tablet_service.cc:1511] Processing CreateTablet for tablet 8059d2e14ad344088a88501e2ba66ebc (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.657876 14840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8059d2e14ad344088a88501e2ba66ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.662532 15129 tablet_service.cc:1511] Processing CreateTablet for tablet 8059d2e14ad344088a88501e2ba66ebc (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.663837 14905 heartbeater.cc:499] Master 127.11.233.253:39245 was elected leader, sending a full tablet report...
I20260926 19:59:36.665943 15129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8059d2e14ad344088a88501e2ba66ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.685087 15194 heartbeater.cc:499] Master 127.11.233.253:39245 was elected leader, sending a full tablet report...
19:59:36.696 [HMS - ERROR - pool-6-thread-3] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_131b6490764e47dc8a7a589300bbf272 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
W20260926 19:59:36.703401 14544 catalog_manager.cc:2209] Already present: failed to create HMS catalog entry for table [id=2484e001a5a142238f1b33238d2cf66f]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_131b6490764e47dc8a7a589300bbf272 already exists)
I20260926 19:59:36.708827 14544 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53020:
name: "default.Table_2369b73124a04219b46431eeadde2ac7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:36.727011 15252 tablet_bootstrap.cc:492] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:36.733207 15050 heartbeater.cc:499] Master 127.11.233.253:39245 was elected leader, sending a full tablet report...
I20260926 19:59:36.739337 15255 tablet_bootstrap.cc:492] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:36.741132 15252 tablet_bootstrap.cc:654] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:36.743741 15252 log.cc:826] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: Log is configured to *not* fsync() on all Append() calls
I20260926 19:59:36.748461 15254 tablet_bootstrap.cc:492] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:36.754060 14542 master_service.cc:960] ReplaceTablet: received request to replace tablet 8059d2e14ad344088a88501e2ba66ebc from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:36.767380 15255 tablet_bootstrap.cc:654] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:36.772845 15254 tablet_bootstrap.cc:654] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:36.773770 14541 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53020:
name: "default.Table_048abbb957574bd9904e0ba925421cee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:36.775615 15252 tablet_bootstrap.cc:492] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:36.776185 15252 ts_tablet_manager.cc:1403] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.050s user 0.013s sys 0.011s
I20260926 19:59:36.776293 15255 log.cc:826] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: Log is configured to *not* fsync() on all Append() calls
I20260926 19:59:36.780594 15254 log.cc:826] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: Log is configured to *not* fsync() on all Append() calls
I20260926 19:59:36.786876 14542 catalog_manager.cc:6133] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Sending DeleteTablet for 3 replicas of tablet 8059d2e14ad344088a88501e2ba66ebc
I20260926 19:59:36.787621 14543 master_service.cc:960] ReplaceTablet: received request to replace tablet 8059d2e14ad344088a88501e2ba66ebc from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:36.789849 14984 tablet_service.cc:1558] Processing DeleteTablet for tablet 8059d2e14ad344088a88501e2ba66ebc with delete_type TABLET_DATA_DELETED (replaced by tablet e0f57c2ced594520a899bfde29795641) from {username='slave'} at 127.0.0.1:43588
I20260926 19:59:36.794566 15254 tablet_bootstrap.cc:492] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:36.794620 15129 tablet_service.cc:1558] Processing DeleteTablet for tablet 8059d2e14ad344088a88501e2ba66ebc with delete_type TABLET_DATA_DELETED (replaced by tablet e0f57c2ced594520a899bfde29795641) from {username='slave'} at 127.0.0.1:38672
I20260926 19:59:36.795161 15254 ts_tablet_manager.cc:1403] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.004s
I20260926 19:59:36.804426 15255 tablet_bootstrap.cc:492] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:36.804962 15255 ts_tablet_manager.cc:1403] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.066s user 0.010s sys 0.003s
I20260926 19:59:36.822922 14545 catalog_manager.cc:2213] created HMS catalog entry for table [id=35c3d430e5a545678edb506779d24bdb]
W20260926 19:59:36.823319 14534 catalog_manager.cc:5043] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): delete failed for tablet 8059d2e14ad344088a88501e2ba66ebc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8059d2e14ad344088a88501e2ba66ebc already in progress: creating tablet
I20260926 19:59:36.826335 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.254:32801
--webserver_interface=127.11.233.254
--webserver_port=45057
--builtin_ntp_servers=127.11.233.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--hive_metastore_uris=thrift://127.0.0.1:43737
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 19:59:36.832603 15252 raft_consensus.cc:359] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:36.833674 15252 raft_consensus.cc:385] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:36.834018 15252 raft_consensus.cc:740] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:36.835462 14839 tablet_service.cc:1558] Processing DeleteTablet for tablet 8059d2e14ad344088a88501e2ba66ebc with delete_type TABLET_DATA_DELETED (replaced by tablet e0f57c2ced594520a899bfde29795641) from {username='slave'} at 127.0.0.1:60562
I20260926 19:59:36.835170 15252 consensus_queue.cc:260] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:36.840735 14838 tablet_service.cc:1511] Processing CreateTablet for tablet e0f57c2ced594520a899bfde29795641 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.842132 14838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0f57c2ced594520a899bfde29795641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.845607 15252 ts_tablet_manager.cc:1434] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.069s user 0.031s sys 0.010s
I20260926 19:59:36.853945 14542 catalog_manager.cc:6934] ReplaceTablet: tablet 8059d2e14ad344088a88501e2ba66ebc deleted and replaced by tablet e0f57c2ced594520a899bfde29795641
W20260926 19:59:36.864120 14534 catalog_manager.cc:5043] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): delete failed for tablet 8059d2e14ad344088a88501e2ba66ebc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8059d2e14ad344088a88501e2ba66ebc already in progress: creating tablet
W20260926 19:59:36.869026 14532 catalog_manager.cc:5043] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): delete failed for tablet 8059d2e14ad344088a88501e2ba66ebc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8059d2e14ad344088a88501e2ba66ebc already in progress: creating tablet
I20260926 19:59:36.869153 14984 tablet_service.cc:1511] Processing CreateTablet for tablet e0f57c2ced594520a899bfde29795641 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.870298 14984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0f57c2ced594520a899bfde29795641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.891788 15129 tablet_service.cc:1558] Processing DeleteTablet for tablet 8059d2e14ad344088a88501e2ba66ebc with delete_type TABLET_DATA_DELETED (replaced by tablet e0f57c2ced594520a899bfde29795641) from {username='slave'} at 127.0.0.1:38672
I20260926 19:59:36.892791 15261 tablet_replica.cc:333] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:36.892573 15128 tablet_service.cc:1511] Processing CreateTablet for tablet e0f57c2ced594520a899bfde29795641 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.893703 15261 raft_consensus.cc:2243] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:36.893834 15128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0f57c2ced594520a899bfde29795641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.894341 15261 raft_consensus.cc:2272] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:36.899823 15261 ts_tablet_manager.cc:1916] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:36.902431 15255 raft_consensus.cc:359] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:36.903402 15255 raft_consensus.cc:385] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:36.903707 15255 raft_consensus.cc:740] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:36.904520 14545 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53020:
table { table_name: "default.Table_7a1c4598445a4ac5845670a68237cb86" } new_table_name: "default.Table_2d6e52c75c4f4413a0185f462c80af99" modify_external_catalogs: true
I20260926 19:59:36.907127 15254 raft_consensus.cc:359] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:36.908038 15254 raft_consensus.cc:385] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:36.908339 15254 raft_consensus.cc:740] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:36.909247 15254 consensus_queue.cc:260] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:36.917793 15255 consensus_queue.cc:260] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:36.927464 15254 ts_tablet_manager.cc:1434] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.132s user 0.031s sys 0.004s
I20260926 19:59:36.934149 15261 ts_tablet_manager.cc:1929] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:36.934625 15261 log.cc:1199] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/8059d2e14ad344088a88501e2ba66ebc
I20260926 19:59:36.935588 15261 ts_tablet_manager.cc:1950] T 8059d2e14ad344088a88501e2ba66ebc P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:36.937193 15252 tablet_bootstrap.cc:492] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:36.937927 15254 tablet_bootstrap.cc:492] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:36.939091 14983 tablet_service.cc:1558] Processing DeleteTablet for tablet 8059d2e14ad344088a88501e2ba66ebc with delete_type TABLET_DATA_DELETED (replaced by tablet e0f57c2ced594520a899bfde29795641) from {username='slave'} at 127.0.0.1:43588
W20260926 19:59:36.940968 14532 catalog_manager.cc:5043] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): delete failed for tablet 8059d2e14ad344088a88501e2ba66ebc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8059d2e14ad344088a88501e2ba66ebc already in progress: creating tablet
I20260926 19:59:36.948690 15255 ts_tablet_manager.cc:1434] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.143s user 0.030s sys 0.009s
I20260926 19:59:36.948740 14838 tablet_service.cc:1558] Processing DeleteTablet for tablet 8059d2e14ad344088a88501e2ba66ebc with delete_type TABLET_DATA_DELETED (replaced by tablet e0f57c2ced594520a899bfde29795641) from {username='slave'} at 127.0.0.1:60562
I20260926 19:59:36.949836 15264 tablet_replica.cc:333] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:36.950721 15264 raft_consensus.cc:2243] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:36.951319 15264 raft_consensus.cc:2272] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:36.951930 15252 tablet_bootstrap.cc:654] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:36.959138 15264 ts_tablet_manager.cc:1916] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:36.961987 14838 tablet_service.cc:1511] Processing CreateTablet for tablet 40adc41045fb4a319739358b468653dd (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.963322 14838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40adc41045fb4a319739358b468653dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.964546 14534 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet 8059d2e14ad344088a88501e2ba66ebc (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:36.968760 14984 tablet_service.cc:1558] Processing DeleteTablet for tablet 8059d2e14ad344088a88501e2ba66ebc with delete_type TABLET_DATA_DELETED (replaced by tablet e0f57c2ced594520a899bfde29795641) from {username='slave'} at 127.0.0.1:43588
I20260926 19:59:36.969736 15260 tablet_replica.cc:333] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:36.970572 15260 raft_consensus.cc:2243] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:36.971158 15260 raft_consensus.cc:2272] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:36.973830 14983 tablet_service.cc:1511] Processing CreateTablet for tablet 40adc41045fb4a319739358b468653dd (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.975003 15260 ts_tablet_manager.cc:1916] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:36.975014 14983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40adc41045fb4a319739358b468653dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.977087 15255 tablet_bootstrap.cc:492] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:36.984383 15254 tablet_bootstrap.cc:654] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:36.990046 15255 tablet_bootstrap.cc:654] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:36.990023 15252 tablet_bootstrap.cc:492] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:36.990470 15252 ts_tablet_manager.cc:1403] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.053s user 0.014s sys 0.006s
I20260926 19:59:36.990335 15128 tablet_service.cc:1511] Processing CreateTablet for tablet 40adc41045fb4a319739358b468653dd (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:36.991321 15128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40adc41045fb4a319739358b468653dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:36.994047 15252 raft_consensus.cc:359] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:36.994769 15252 raft_consensus.cc:385] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:36.995044 15252 raft_consensus.cc:740] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:36.995971 15252 consensus_queue.cc:260] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:36.996950 15264 ts_tablet_manager.cc:1929] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.000016 15254 tablet_bootstrap.cc:492] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:37.000466 15254 ts_tablet_manager.cc:1403] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.063s user 0.006s sys 0.004s
I20260926 19:59:37.000988 14542 master_service.cc:960] ReplaceTablet: received request to replace tablet e0f57c2ced594520a899bfde29795641 from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:37.003305 15254 raft_consensus.cc:359] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.003999 15254 raft_consensus.cc:385] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.004276 15254 raft_consensus.cc:740] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:36.997355 15264 log.cc:1199] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/8059d2e14ad344088a88501e2ba66ebc
I20260926 19:59:37.005224 15252 ts_tablet_manager.cc:1434] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260926 19:59:37.005730 15264 ts_tablet_manager.cc:1950] T 8059d2e14ad344088a88501e2ba66ebc P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:37.009197 14534 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet 8059d2e14ad344088a88501e2ba66ebc (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:37.010049 15254 consensus_queue.cc:260] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.012403 15254 ts_tablet_manager.cc:1434] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.012s user 0.007s sys 0.001s
I20260926 19:59:37.013957 15254 tablet_bootstrap.cc:492] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:37.021481 15254 tablet_bootstrap.cc:654] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.050545 15254 tablet_bootstrap.cc:492] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:37.050967 15254 ts_tablet_manager.cc:1403] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.037s user 0.007s sys 0.007s
I20260926 19:59:37.053397 15254 raft_consensus.cc:359] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.054117 15254 raft_consensus.cc:385] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.054382 15254 raft_consensus.cc:740] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.054569 14542 catalog_manager.cc:6133] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Sending DeleteTablet for 3 replicas of tablet e0f57c2ced594520a899bfde29795641
I20260926 19:59:37.055321 15254 consensus_queue.cc:260] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.057663 15254 ts_tablet_manager.cc:1434] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260926 19:59:37.060155 15255 tablet_bootstrap.cc:492] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:37.060649 15255 ts_tablet_manager.cc:1403] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.089s user 0.020s sys 0.001s
I20260926 19:59:37.063917 15255 raft_consensus.cc:359] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.064709 15255 raft_consensus.cc:385] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.065006 15255 raft_consensus.cc:740] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.065961 15255 consensus_queue.cc:260] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.069176 14542 catalog_manager.cc:6934] ReplaceTablet: tablet e0f57c2ced594520a899bfde29795641 deleted and replaced by tablet be4749d613b341c8879cd827e9ef27cc
I20260926 19:59:37.060184 14983 tablet_service.cc:1558] Processing DeleteTablet for tablet e0f57c2ced594520a899bfde29795641 with delete_type TABLET_DATA_DELETED (replaced by tablet be4749d613b341c8879cd827e9ef27cc) from {username='slave'} at 127.0.0.1:43588
I20260926 19:59:37.072804 15255 ts_tablet_manager.cc:1434] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.012s user 0.003s sys 0.004s
I20260926 19:59:37.081221 14539 master_service.cc:960] ReplaceTablet: received request to replace tablet 40adc41045fb4a319739358b468653dd from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:37.087348 14543 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53020:
name: "default.Table_c74a17cf08a9482ab2915e2d7ff9fbeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:37.089650 15252 tablet_bootstrap.cc:492] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:37.089982 15255 tablet_bootstrap.cc:492] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:37.097431 15252 tablet_bootstrap.cc:654] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.098857 15260 ts_tablet_manager.cc:1929] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.099293 15260 log.cc:1199] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/8059d2e14ad344088a88501e2ba66ebc
I20260926 19:59:37.103294 15260 ts_tablet_manager.cc:1950] T 8059d2e14ad344088a88501e2ba66ebc P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
W20260926 19:59:37.105055 14913 tablet.cc:2433] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:59:37.106390 15260 tablet_replica.cc:333] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:37.106683 14532 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet 8059d2e14ad344088a88501e2ba66ebc (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:37.107010 15260 raft_consensus.cc:2243] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260926 19:59:37.105685 14534 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.252:37917, user_credentials={real_user=slave}} blocked reactor thread for 55942.4us
I20260926 19:59:37.107457 15260 raft_consensus.cc:2272] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.108860 14838 tablet_service.cc:1558] Processing DeleteTablet for tablet e0f57c2ced594520a899bfde29795641 with delete_type TABLET_DATA_DELETED (replaced by tablet be4749d613b341c8879cd827e9ef27cc) from {username='slave'} at 127.0.0.1:60562
I20260926 19:59:37.110039 15129 tablet_service.cc:1558] Processing DeleteTablet for tablet e0f57c2ced594520a899bfde29795641 with delete_type TABLET_DATA_DELETED (replaced by tablet be4749d613b341c8879cd827e9ef27cc) from {username='slave'} at 127.0.0.1:38672
I20260926 19:59:37.110208 15255 tablet_bootstrap.cc:654] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.111024 15261 tablet_replica.cc:333] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:37.111768 15261 raft_consensus.cc:2243] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.112315 15261 raft_consensus.cc:2272] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.113238 15264 tablet_replica.cc:333] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:37.114117 15264 raft_consensus.cc:2243] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.114683 15264 raft_consensus.cc:2272] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.117578 15261 ts_tablet_manager.cc:1916] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.119483 15264 ts_tablet_manager.cc:1916] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.124599 15267 raft_consensus.cc:493] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:37.125172 15267 raft_consensus.cc:515] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.127498 15128 tablet_service.cc:1558] Processing DeleteTablet for tablet e0f57c2ced594520a899bfde29795641 with delete_type TABLET_DATA_DELETED (replaced by tablet be4749d613b341c8879cd827e9ef27cc) from {username='slave'} at 127.0.0.1:38672
I20260926 19:59:37.129875 15267 leader_election.cc:290] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:37.145742 15260 ts_tablet_manager.cc:1916] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.153260 14539 catalog_manager.cc:6133] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Sending DeleteTablet for 3 replicas of tablet 40adc41045fb4a319739358b468653dd
I20260926 19:59:37.154934 14539 catalog_manager.cc:6934] ReplaceTablet: tablet 40adc41045fb4a319739358b468653dd deleted and replaced by tablet 087ff41f2a8c44e09389a5dd862ebbc4
I20260926 19:59:37.161768 14983 tablet_service.cc:1558] Processing DeleteTablet for tablet 40adc41045fb4a319739358b468653dd with delete_type TABLET_DATA_DELETED (replaced by tablet 087ff41f2a8c44e09389a5dd862ebbc4) from {username='slave'} at 127.0.0.1:43588
I20260926 19:59:37.164772 15128 tablet_service.cc:1558] Processing DeleteTablet for tablet 40adc41045fb4a319739358b468653dd with delete_type TABLET_DATA_DELETED (replaced by tablet 087ff41f2a8c44e09389a5dd862ebbc4) from {username='slave'} at 127.0.0.1:38672
I20260926 19:59:37.166952 15255 tablet_bootstrap.cc:492] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:37.167354 15255 ts_tablet_manager.cc:1403] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.094s user 0.007s sys 0.003s
I20260926 19:59:37.168840 14838 tablet_service.cc:1558] Processing DeleteTablet for tablet 40adc41045fb4a319739358b468653dd with delete_type TABLET_DATA_DELETED (replaced by tablet 087ff41f2a8c44e09389a5dd862ebbc4) from {username='slave'} at 127.0.0.1:60562
W20260926 19:59:37.181206 14538 catalog_manager.cc:5671] Ignoring report from unknown tablet e0f57c2ced594520a899bfde29795641
I20260926 19:59:37.185015 15252 tablet_bootstrap.cc:492] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:37.188409 14546 catalog_manager.cc:2213] created HMS catalog entry for table [id=ed2f37ba144d40c5a4751dc80b67c792]
I20260926 19:59:37.185474 15252 ts_tablet_manager.cc:1403] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.096s user 0.007s sys 0.010s
I20260926 19:59:37.194017 15255 raft_consensus.cc:359] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.194765 15255 raft_consensus.cc:385] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.195039 15255 raft_consensus.cc:740] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.196588 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40adc41045fb4a319739358b468653dd" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" is_pre_election: true
I20260926 19:59:37.197242 15252 raft_consensus.cc:359] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.209703 15252 raft_consensus.cc:385] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.210001 15252 raft_consensus.cc:740] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.210161 15255 consensus_queue.cc:260] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
F20260926 19:59:37.210181 14722 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1790452776, eventType=CREATE_TABLE, dbName=default, tableName=table_131b6490764e47dc8a7a589300bbf272, message={"server":"","servicePrincipal":"","db":"default","table":"table_131b6490764e47dc8a7a589300bbf272","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_131b6490764e47dc8a7a589300bbf272\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790452775},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/warehouse/table_131b6490764e47dc8a7a589300bbf272\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_131b6490764e47dc8a7a589300bbf272\",\"kudu.table_id\":\"53eb5bec635046daa62c8b441e0d404c\",\"transient_lastDdlTime\":\"1790452775\",\"comment\":\"\",\"kudu.master_addresses\":\"127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"b4293131cc434d2194a1cdf2d39da0b5\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790452776,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1790452776, eventType=CREATE_TABLE, dbName=default, tableName=table_214fa130d427469eaf37415c0767c7e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_214fa130d427469eaf37415c0767c7e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_214fa130d427469eaf37415c0767c7e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790452776},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/warehouse/table_214fa130d427469eaf37415c0767c7e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_214fa130d427469eaf37415c0767c7e3\",\"kudu.table_id\":\"a07796662b5d4b0b8b019aa4e111b875\",\"transient_lastDdlTime\":\"1790452776\",\"comment\":\"\",\"kudu.master_addresses\":\"127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"b4293131cc434d2194a1cdf2d39da0b5\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790452776,"files":[]}, messageFormat=json-0.2, catName=hive);
I20260926 19:59:37.212172 15255 ts_tablet_manager.cc:1434] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.045s user 0.006s sys 0.000s
*** Check failure stack trace: ***
I20260926 19:59:37.219197 15260 ts_tablet_manager.cc:1929] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.212867 14546 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:53020 (request call id 18) took 1074 ms. Trace:
I20260926 19:59:37.219573 15260 log.cc:1199] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/e0f57c2ced594520a899bfde29795641
I20260926 19:59:37.220228 15260 ts_tablet_manager.cc:1950] T e0f57c2ced594520a899bfde29795641 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
I20260926 19:59:37.219681 14546 rpcz_store.cc:276] 0926 19:59:36.138929 (+ 0us) service_pool.cc:167] Inserting onto call queue
0926 19:59:36.139044 (+ 115us) service_pool.cc:224] Handling call
0926 19:59:36.456767 (+317723us) catalog_manager.cc:2140] Acquired catalog manager lock
0926 19:59:36.458614 (+ 1847us) catalog_manager.cc:2186] Created new table and tablet info
0926 19:59:37.188364 (+729750us) catalog_manager.cc:2212] Created new table in HMS catalog
0926 19:59:37.212128 (+ 23764us) catalog_manager.cc:2246] Wrote table and tablets to system table
0926 19:59:37.212479 (+ 351us) catalog_manager.cc:2280] Inserted table and tablets into CatalogManager maps
0926 19:59:37.212792 (+ 313us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0926 19:59:37.190513 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0926 19:59:37.191004 (+ 491us) write_op.cc:432] Acquiring schema lock in shared mode
0926 19:59:37.191031 (+ 27us) write_op.cc:435] Acquired schema lock
0926 19:59:37.191047 (+ 16us) tablet.cc:662] Decoding operations
0926 19:59:37.191355 (+ 308us) write_op.cc:620] Acquiring the partition lock for write op
0926 19:59:37.191400 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0926 19:59:37.191431 (+ 31us) tablet.cc:685] Acquiring locks for 2 operations
0926 19:59:37.191670 (+ 239us) tablet.cc:701] Row locks acquired
0926 19:59:37.191688 (+ 18us) write_op.cc:260] PREPARE: finished
0926 19:59:37.191819 (+ 131us) write_op.cc:270] Start()
0926 19:59:37.191908 (+ 89us) write_op.cc:276] Timestamp: P: 1790452777191761 usec, L: 0
0926 19:59:37.191926 (+ 18us) op_driver.cc:348] REPLICATION: starting
0926 19:59:37.192445 (+ 519us) log.cc:844] Serialized 432 byte log entry
0926 19:59:37.206343 (+ 13898us) op_driver.cc:464] REPLICATION: finished
0926 19:59:37.206634 (+ 291us) write_op.cc:301] APPLY: starting
0926 19:59:37.206693 (+ 59us) tablet.cc:1394] starting BulkCheckPresence
0926 19:59:37.206807 (+ 114us) tablet.cc:1397] finished BulkCheckPresence
0926 19:59:37.206909 (+ 102us) tablet.cc:1399] starting ApplyRowOperation cycle
0926 19:59:37.207291 (+ 382us) tablet.cc:1410] finished ApplyRowOperation cycle
0926 19:59:37.207434 (+ 143us) tablet_metrics.cc:600] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0926 19:59:37.207479 (+ 45us) write_op.cc:312] APPLY: finished
0926 19:59:37.207688 (+ 209us) log.cc:844] Serialized 28 byte log entry
0926 19:59:37.208444 (+ 756us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:59:37.208513 (+ 69us) write_op.cc:454] Released schema lock
0926 19:59:37.208561 (+ 48us) write_op.cc:341] FINISH: Updating metrics
0926 19:59:37.208820 (+ 259us) write_op.cc:489] Releasing partition, row and schema locks
0926 19:59:37.208856 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":363051,"Hive Metastore.run_cpu_time_us":733,"Hive Metastore.run_wall_time_us":365819,"child_traces":[["op",{"apply.queue_time_us":96,"apply.run_cpu_time_us":1563,"apply.run_wall_time_us":2160,"num_ops":2,"prepare.queue_time_us":136,"prepare.run_cpu_time_us":2259,"prepare.run_wall_time_us":2259,"raft.queue_time_us":203,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":11,"replication_time_us":14309}]]}
I20260926 19:59:37.226542 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40adc41045fb4a319739358b468653dd" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" is_pre_election: true
I20260926 19:59:37.227398 15004 raft_consensus.cc:2468] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:37.229226 14793 leader_election.cc:304] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:37.229895 15260 tablet_replica.cc:333] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:37.229905 14532 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet e0f57c2ced594520a899bfde29795641 (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:37.230540 15260 raft_consensus.cc:2243] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.230603 15267 raft_consensus.cc:2804] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:37.230948 15267 raft_consensus.cc:493] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:37.231027 15260 raft_consensus.cc:2272] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.231287 15267 raft_consensus.cc:3060] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:37.238266 15267 raft_consensus.cc:515] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.229609 14546 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53020:
table { table_name: "default.Table_e144b6535e8448a3b04ecd97e88b06a9" } modify_external_catalogs: true
I20260926 19:59:37.241115 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40adc41045fb4a319739358b468653dd" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
I20260926 19:59:37.241660 15148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40adc41045fb4a319739358b468653dd" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
I20260926 19:59:37.241376 14546 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53020:
table { table_name: "default.Table_e144b6535e8448a3b04ecd97e88b06a9" } modify_external_catalogs: true
I20260926 19:59:37.242628 15267 leader_election.cc:290] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Requested vote from peers 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
W20260926 19:59:37.242730 14793 leader_election.cc:343] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:37.246060 14793 leader_election.cc:304] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56196546dad349b9bd344c4e0b6a6874; no voters: 17fec3df2f014ed5b1803b8a2bc38df9, e3e2a17ea4c94765be709a7ec8af4c8a
I20260926 19:59:37.249130 15260 ts_tablet_manager.cc:1916] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.249583 15267 raft_consensus.cc:2749] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 19:59:37.253607 15252 consensus_queue.cc:260] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.254717 15149 raft_consensus.cc:2468] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:37.255746 15252 ts_tablet_manager.cc:1434] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.062s user 0.005s sys 0.000s
I20260926 19:59:37.282560 15260 ts_tablet_manager.cc:1929] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.282950 15260 log.cc:1199] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/40adc41045fb4a319739358b468653dd
I20260926 19:59:37.283627 15260 ts_tablet_manager.cc:1950] T 40adc41045fb4a319739358b468653dd P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
I20260926 19:59:37.284313 14540 master_service.cc:960] ReplaceTablet: received request to replace tablet e0f57c2ced594520a899bfde29795641 from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:37.286700 14532 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet 40adc41045fb4a319739358b468653dd (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:37.287348 15264 ts_tablet_manager.cc:1929] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.287753 15264 log.cc:1199] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/e0f57c2ced594520a899bfde29795641
I20260926 19:59:37.288396 15264 ts_tablet_manager.cc:1950] T e0f57c2ced594520a899bfde29795641 P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:37.288481 14983 tablet_service.cc:1511] Processing CreateTablet for tablet 087ff41f2a8c44e09389a5dd862ebbc4 (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.289592 14983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087ff41f2a8c44e09389a5dd862ebbc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.290083 15128 tablet_service.cc:1511] Processing CreateTablet for tablet 087ff41f2a8c44e09389a5dd862ebbc4 (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.290287 15261 ts_tablet_manager.cc:1929] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.290676 15261 log.cc:1199] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/e0f57c2ced594520a899bfde29795641
I20260926 19:59:37.291136 15128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087ff41f2a8c44e09389a5dd862ebbc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.291309 15261 ts_tablet_manager.cc:1950] T e0f57c2ced594520a899bfde29795641 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:37.293088 14838 tablet_service.cc:1511] Processing CreateTablet for tablet 087ff41f2a8c44e09389a5dd862ebbc4 (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.294355 14838 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 087ff41f2a8c44e09389a5dd862ebbc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.294919 14534 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet e0f57c2ced594520a899bfde29795641 (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:37.301597 15264 tablet_replica.cc:333] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:37.301776 14534 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet e0f57c2ced594520a899bfde29795641 (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:37.302266 15264 raft_consensus.cc:2243] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.302718 15264 raft_consensus.cc:2272] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.303910 15261 tablet_replica.cc:333] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
W20260926 19:59:37.304005 14534 catalog_manager.cc:5030] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): delete failed for tablet e0f57c2ced594520a899bfde29795641 because the tablet was not found. No further retry: Not found: Tablet not found: e0f57c2ced594520a899bfde29795641
I20260926 19:59:37.304538 15261 raft_consensus.cc:2243] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.305012 15261 raft_consensus.cc:2272] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.307691 14840 tablet_service.cc:1511] Processing CreateTablet for tablet be4749d613b341c8879cd827e9ef27cc (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.308833 14840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4749d613b341c8879cd827e9ef27cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.311631 14984 tablet_service.cc:1511] Processing CreateTablet for tablet be4749d613b341c8879cd827e9ef27cc (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.312819 14984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4749d613b341c8879cd827e9ef27cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.315095 15261 ts_tablet_manager.cc:1916] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.327204 15252 tablet_bootstrap.cc:492] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:37.327492 15129 tablet_service.cc:1511] Processing CreateTablet for tablet be4749d613b341c8879cd827e9ef27cc (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.328606 15129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4749d613b341c8879cd827e9ef27cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.340238 15252 tablet_bootstrap.cc:654] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
@ 0x7fe3fa162920 google::LogMessage::Flush() at ??:0
I20260926 19:59:37.362265 15264 ts_tablet_manager.cc:1916] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.370493 14540 master_service.cc:960] ReplaceTablet: received request to replace tablet 40adc41045fb4a319739358b468653dd from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:37.374884 15254 tablet_bootstrap.cc:492] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:37.376838 14982 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8337dd2c7440c08ca282be7496806e (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.378140 14982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8337dd2c7440c08ca282be7496806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.378842 15128 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8337dd2c7440c08ca282be7496806e (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.378844 14839 tablet_service.cc:1511] Processing CreateTablet for tablet 0c8337dd2c7440c08ca282be7496806e (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.379902 15128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8337dd2c7440c08ca282be7496806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.380052 14839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8337dd2c7440c08ca282be7496806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.385628 15255 tablet_bootstrap.cc:492] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:37.387210 15254 tablet_bootstrap.cc:654] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.391526 15252 tablet_bootstrap.cc:492] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:37.391956 15252 ts_tablet_manager.cc:1403] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.065s user 0.001s sys 0.009s
I20260926 19:59:37.396283 15255 tablet_bootstrap.cc:654] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.397538 15252 raft_consensus.cc:359] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.407012 15252 raft_consensus.cc:385] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.407305 15252 raft_consensus.cc:740] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.408804 15264 ts_tablet_manager.cc:1929] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.409188 15264 log.cc:1199] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/40adc41045fb4a319739358b468653dd
I20260926 19:59:37.409893 15264 ts_tablet_manager.cc:1950] T 40adc41045fb4a319739358b468653dd P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:37.410786 15252 consensus_queue.cc:260] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.412379 15254 tablet_bootstrap.cc:492] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:37.412724 15252 ts_tablet_manager.cc:1434] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260926 19:59:37.412616 14534 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet 40adc41045fb4a319739358b468653dd (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:37.412802 15254 ts_tablet_manager.cc:1403] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.010s
I20260926 19:59:37.415688 15254 raft_consensus.cc:359] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.416366 15254 raft_consensus.cc:385] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.416646 15254 raft_consensus.cc:740] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
@ 0x7fe3fa16268f google::LogMessage::~LogMessage() at ??:0
I20260926 19:59:37.417948 15252 tablet_bootstrap.cc:492] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:37.417358 15254 consensus_queue.cc:260] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.422329 15261 ts_tablet_manager.cc:1929] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.422721 15261 log.cc:1199] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/40adc41045fb4a319739358b468653dd
I20260926 19:59:37.425005 15254 ts_tablet_manager.cc:1434] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.012s user 0.003s sys 0.004s
I20260926 19:59:37.423552 15261 ts_tablet_manager.cc:1950] T 40adc41045fb4a319739358b468653dd P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:37.431011 14539 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53020:
name: "default.Table_3c580a21625b41a4894dd8072db5ba09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:37.431659 15254 tablet_bootstrap.cc:492] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:37.432325 14534 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet 40adc41045fb4a319739358b468653dd (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:37.437388 15252 tablet_bootstrap.cc:654] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.437860 15254 tablet_bootstrap.cc:654] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.443989 15252 tablet_bootstrap.cc:492] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:37.444649 15252 ts_tablet_manager.cc:1403] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.031s user 0.011s sys 0.002s
I20260926 19:59:37.445744 15255 tablet_bootstrap.cc:492] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:37.446187 15255 ts_tablet_manager.cc:1403] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.065s user 0.005s sys 0.006s
I20260926 19:59:37.447965 15252 raft_consensus.cc:359] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.449169 15252 raft_consensus.cc:385] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.449532 15252 raft_consensus.cc:740] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.450582 15252 consensus_queue.cc:260] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.456754 15252 ts_tablet_manager.cc:1434] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260926 19:59:37.457914 15252 tablet_bootstrap.cc:492] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:37.449074 15255 raft_consensus.cc:359] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.483085 15255 raft_consensus.cc:385] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.483420 15255 raft_consensus.cc:740] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.484332 15255 consensus_queue.cc:260] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.485088 15252 tablet_bootstrap.cc:654] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.500212 15254 tablet_bootstrap.cc:492] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:37.500734 15254 ts_tablet_manager.cc:1403] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.069s user 0.022s sys 0.039s
I20260926 19:59:37.503772 14540 master_service.cc:960] ReplaceTablet: received request to replace tablet be4749d613b341c8879cd827e9ef27cc from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:37.504212 15254 raft_consensus.cc:359] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.504935 15254 raft_consensus.cc:385] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.505211 15254 raft_consensus.cc:740] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.514945 15255 ts_tablet_manager.cc:1434] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.068s user 0.001s sys 0.006s
I20260926 19:59:37.522876 15252 tablet_bootstrap.cc:492] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:37.523330 15252 ts_tablet_manager.cc:1403] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.066s user 0.012s sys 0.015s
I20260926 19:59:37.530108 15255 tablet_bootstrap.cc:492] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:37.527175 15252 raft_consensus.cc:359] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:37.531213 15252 raft_consensus.cc:385] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.531683 15252 raft_consensus.cc:740] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.535615 14540 catalog_manager.cc:6133] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Sending DeleteTablet for 3 replicas of tablet be4749d613b341c8879cd827e9ef27cc
I20260926 19:59:37.532572 15252 consensus_queue.cc:260] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:37.537014 15255 tablet_bootstrap.cc:654] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.538439 14839 tablet_service.cc:1558] Processing DeleteTablet for tablet be4749d613b341c8879cd827e9ef27cc with delete_type TABLET_DATA_DELETED (replaced by tablet 21847f1f910b458fad550d9ef426173e) from {username='slave'} at 127.0.0.1:60562
I20260926 19:59:37.545991 15254 consensus_queue.cc:260] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
W20260926 19:59:37.546530 14534 catalog_manager.cc:5043] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): delete failed for tablet be4749d613b341c8879cd827e9ef27cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet be4749d613b341c8879cd827e9ef27cc already in progress: creating tablet
I20260926 19:59:37.548696 15254 ts_tablet_manager.cc:1434] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.048s user 0.007s sys 0.000s
I20260926 19:59:37.549758 15254 tablet_bootstrap.cc:492] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:37.553635 15252 ts_tablet_manager.cc:1434] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.030s user 0.007s sys 0.000s
I20260926 19:59:37.558848 15254 tablet_bootstrap.cc:654] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.566493 14839 tablet_service.cc:1558] Processing DeleteTablet for tablet be4749d613b341c8879cd827e9ef27cc with delete_type TABLET_DATA_DELETED (replaced by tablet 21847f1f910b458fad550d9ef426173e) from {username='slave'} at 127.0.0.1:60562
I20260926 19:59:37.568347 14540 catalog_manager.cc:6934] ReplaceTablet: tablet be4749d613b341c8879cd827e9ef27cc deleted and replaced by tablet 21847f1f910b458fad550d9ef426173e
@ 0x7fe4052f535c kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 19:59:37.575263 14982 tablet_service.cc:1558] Processing DeleteTablet for tablet be4749d613b341c8879cd827e9ef27cc with delete_type TABLET_DATA_DELETED (replaced by tablet 21847f1f910b458fad550d9ef426173e) from {username='slave'} at 127.0.0.1:43588
I20260926 19:59:37.577371 15264 tablet_replica.cc:333] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:37.578150 15264 raft_consensus.cc:2243] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.578646 15264 raft_consensus.cc:2272] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260926 19:59:37.579819 14532 catalog_manager.cc:5043] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): delete failed for tablet be4749d613b341c8879cd827e9ef27cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet be4749d613b341c8879cd827e9ef27cc already in progress: creating tablet
I20260926 19:59:37.583554 15254 tablet_bootstrap.cc:492] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:37.584010 15254 ts_tablet_manager.cc:1403] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.003s
I20260926 19:59:37.586947 15254 raft_consensus.cc:359] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:37.587668 15254 raft_consensus.cc:385] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.587946 15254 raft_consensus.cc:740] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.588143 15128 tablet_service.cc:1558] Processing DeleteTablet for tablet be4749d613b341c8879cd827e9ef27cc with delete_type TABLET_DATA_DELETED (replaced by tablet 21847f1f910b458fad550d9ef426173e) from {username='slave'} at 127.0.0.1:38672
I20260926 19:59:37.590812 15255 tablet_bootstrap.cc:492] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:37.591339 15255 ts_tablet_manager.cc:1403] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.061s user 0.006s sys 0.006s
I20260926 19:59:37.592002 15261 tablet_replica.cc:333] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:37.592778 15261 raft_consensus.cc:2243] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.593257 15261 raft_consensus.cc:2272] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.594237 15264 ts_tablet_manager.cc:1916] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.595371 15255 raft_consensus.cc:359] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.596038 15255 raft_consensus.cc:385] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.596300 15255 raft_consensus.cc:740] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.593186 15254 consensus_queue.cc:260] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:37.596966 15255 consensus_queue.cc:260] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.598994 15261 ts_tablet_manager.cc:1916] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.599334 15254 ts_tablet_manager.cc:1434] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260926 19:59:37.606180 14540 master_service.cc:960] ReplaceTablet: received request to replace tablet 087ff41f2a8c44e09389a5dd862ebbc4 from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:37.613065 14982 tablet_service.cc:1511] Processing CreateTablet for tablet 21847f1f910b458fad550d9ef426173e (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.613050 14839 tablet_service.cc:1511] Processing CreateTablet for tablet 21847f1f910b458fad550d9ef426173e (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.614203 14839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21847f1f910b458fad550d9ef426173e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.614202 14982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21847f1f910b458fad550d9ef426173e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.623895 15128 tablet_service.cc:1511] Processing CreateTablet for tablet 21847f1f910b458fad550d9ef426173e (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.625032 15128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21847f1f910b458fad550d9ef426173e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.628460 15255 ts_tablet_manager.cc:1434] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.037s user 0.001s sys 0.005s
I20260926 19:59:37.630323 15255 tablet_bootstrap.cc:492] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:37.633167 15266 raft_consensus.cc:493] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:37.633713 15266 raft_consensus.cc:515] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.637039 15266 leader_election.cc:290] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:37.642089 15255 tablet_bootstrap.cc:654] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
@ 0x7fe4052f4b1c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 19:59:37.657016 14540 catalog_manager.cc:6133] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Sending DeleteTablet for 3 replicas of tablet 087ff41f2a8c44e09389a5dd862ebbc4
I20260926 19:59:37.658907 14540 catalog_manager.cc:6934] ReplaceTablet: tablet 087ff41f2a8c44e09389a5dd862ebbc4 deleted and replaced by tablet 5da8e150c6844f9ba6e4a2cb2876fdec
I20260926 19:59:37.659108 14984 tablet_service.cc:1558] Processing DeleteTablet for tablet 087ff41f2a8c44e09389a5dd862ebbc4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5da8e150c6844f9ba6e4a2cb2876fdec) from {username='slave'} at 127.0.0.1:43588
I20260926 19:59:37.659902 15260 tablet_replica.cc:333] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
W20260926 19:59:37.660557 15201 tablet.cc:2433] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:59:37.660619 15260 raft_consensus.cc:2243] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260926 19:59:37.661341 15056 tablet.cc:2433] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:59:37.662297 14840 tablet_service.cc:1558] Processing DeleteTablet for tablet 087ff41f2a8c44e09389a5dd862ebbc4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5da8e150c6844f9ba6e4a2cb2876fdec) from {username='slave'} at 127.0.0.1:60562
I20260926 19:59:37.664270 15129 tablet_service.cc:1558] Processing DeleteTablet for tablet 087ff41f2a8c44e09389a5dd862ebbc4 with delete_type TABLET_DATA_DELETED (replaced by tablet 5da8e150c6844f9ba6e4a2cb2876fdec) from {username='slave'} at 127.0.0.1:38672
I20260926 19:59:37.668459 14984 tablet_service.cc:1558] Processing DeleteTablet for tablet be4749d613b341c8879cd827e9ef27cc with delete_type TABLET_DATA_DELETED (replaced by tablet 21847f1f910b458fad550d9ef426173e) from {username='slave'} at 127.0.0.1:43588
I20260926 19:59:37.672111 15260 raft_consensus.cc:2272] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.674746 15260 ts_tablet_manager.cc:1916] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fe4052f8b92 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260926 19:59:37.682860 15254 tablet_bootstrap.cc:492] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:37.689705 15254 tablet_bootstrap.cc:654] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.700894 15264 ts_tablet_manager.cc:1929] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.701298 15264 log.cc:1199] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/be4749d613b341c8879cd827e9ef27cc
I20260926 19:59:37.702000 15264 ts_tablet_manager.cc:1950] T be4749d613b341c8879cd827e9ef27cc P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:37.704519 14534 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet be4749d613b341c8879cd827e9ef27cc (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:37.704672 15261 ts_tablet_manager.cc:1929] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.705145 15261 log.cc:1199] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/be4749d613b341c8879cd827e9ef27cc
I20260926 19:59:37.705843 15261 ts_tablet_manager.cc:1950] T be4749d613b341c8879cd827e9ef27cc P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:37.717378 15261 tablet_replica.cc:333] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
@ 0x7fe4052f8b4a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260926 19:59:37.719400 15252 tablet_bootstrap.cc:492] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:37.720959 15261 raft_consensus.cc:2243] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.721448 15261 raft_consensus.cc:2272] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.722584 15264 tablet_replica.cc:333] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:37.723357 15264 raft_consensus.cc:2243] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.723857 15264 raft_consensus.cc:2272] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.726330 14534 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet be4749d613b341c8879cd827e9ef27cc (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:37.727285 15252 tablet_bootstrap.cc:654] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.728097 15261 ts_tablet_manager.cc:1916] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.733222 15255 tablet_bootstrap.cc:492] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:37.733680 15255 ts_tablet_manager.cc:1403] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.104s user 0.020s sys 0.017s
I20260926 19:59:37.734297 14839 tablet_service.cc:1511] Processing CreateTablet for tablet 5da8e150c6844f9ba6e4a2cb2876fdec (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.735433 14839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5da8e150c6844f9ba6e4a2cb2876fdec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.736435 15255 raft_consensus.cc:359] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:37.737080 15255 raft_consensus.cc:385] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.737159 15129 tablet_service.cc:1511] Processing CreateTablet for tablet 5da8e150c6844f9ba6e4a2cb2876fdec (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.737329 15255 raft_consensus.cc:740] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.737305 14984 tablet_service.cc:1511] Processing CreateTablet for tablet 5da8e150c6844f9ba6e4a2cb2876fdec (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:37.738276 15129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5da8e150c6844f9ba6e4a2cb2876fdec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.738430 14984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5da8e150c6844f9ba6e4a2cb2876fdec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:37.738095 15255 consensus_queue.cc:260] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:37.743557 15255 ts_tablet_manager.cc:1434] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260926 19:59:37.753355 15264 ts_tablet_manager.cc:1916] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fe4052f8ada std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260926 19:59:37.763288 15254 tablet_bootstrap.cc:492] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:37.763715 15254 ts_tablet_manager.cc:1403] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.081s user 0.007s sys 0.006s
I20260926 19:59:37.765975 15254 raft_consensus.cc:359] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.766598 15254 raft_consensus.cc:385] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.766826 15254 raft_consensus.cc:740] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.770442 15255 tablet_bootstrap.cc:492] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:37.772554 15254 consensus_queue.cc:260] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
W20260926 19:59:37.781633 14940 leader_election.cc:343] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): Not found: Tablet not found: be4749d613b341c8879cd827e9ef27cc
I20260926 19:59:37.783946 15254 ts_tablet_manager.cc:1434] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260926 19:59:37.786729 15255 tablet_bootstrap.cc:654] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.791742 15260 ts_tablet_manager.cc:1929] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.792145 15260 log.cc:1199] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/087ff41f2a8c44e09389a5dd862ebbc4
I20260926 19:59:37.792794 15260 ts_tablet_manager.cc:1950] T 087ff41f2a8c44e09389a5dd862ebbc4 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
I20260926 19:59:37.796509 15252 tablet_bootstrap.cc:492] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
W20260926 19:59:37.796458 14940 leader_election.cc:343] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): Not found: Tablet not found: be4749d613b341c8879cd827e9ef27cc
I20260926 19:59:37.796919 15252 ts_tablet_manager.cc:1403] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.078s user 0.010s sys 0.001s
I20260926 19:59:37.796900 14940 leader_election.cc:304] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9; no voters: 56196546dad349b9bd344c4e0b6a6874, e3e2a17ea4c94765be709a7ec8af4c8a
I20260926 19:59:37.797816 15266 raft_consensus.cc:2749] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
@ 0x7fe4052f8aa2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260926 19:59:37.803177 14542 master_service.cc:960] ReplaceTablet: received request to replace tablet 087ff41f2a8c44e09389a5dd862ebbc4 from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:37.802413 15252 raft_consensus.cc:359] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.803740 15252 raft_consensus.cc:385] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.804003 15252 raft_consensus.cc:740] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.806385 15260 tablet_replica.cc:333] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:37.806654 14532 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet 087ff41f2a8c44e09389a5dd862ebbc4 (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:37.807142 15260 raft_consensus.cc:2243] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:37.807621 15260 raft_consensus.cc:2272] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:37.808979 15255 tablet_bootstrap.cc:492] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:37.809389 15255 ts_tablet_manager.cc:1403] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.011s
I20260926 19:59:37.809989 15252 consensus_queue.cc:260] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.823853 15254 tablet_bootstrap.cc:492] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:37.824828 15252 ts_tablet_manager.cc:1434] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.028s user 0.004s sys 0.002s
@ 0x7fe4052f7d9e std::__1::__function::__func<>::operator()() at ??:0
@ 0x5bd4e5 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5bd31a std::__1::function<>::operator()() at ??:?
I20260926 19:59:37.829275 15254 tablet_bootstrap.cc:654] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.821321 15260 ts_tablet_manager.cc:1916] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:37.837188 15252 tablet_bootstrap.cc:492] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:37.844549 15255 raft_consensus.cc:359] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.845289 15255 raft_consensus.cc:385] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.845573 15255 raft_consensus.cc:740] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.846371 15255 consensus_queue.cc:260] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
@ 0x7fe3fc35881b kudu::Thread::SuperviseThread() at ??:0
@ 0x4569a0 __tsan_thread_start_func at ??:?
I20260926 19:59:37.850842 15252 tablet_bootstrap.cc:654] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.860216 15264 ts_tablet_manager.cc:1929] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.860605 15264 log.cc:1199] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/087ff41f2a8c44e09389a5dd862ebbc4
I20260926 19:59:37.861065 15255 ts_tablet_manager.cc:1434] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260926 19:59:37.861235 15264 ts_tablet_manager.cc:1950] T 087ff41f2a8c44e09389a5dd862ebbc4 P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:37.861982 15261 ts_tablet_manager.cc:1929] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.862349 15261 log.cc:1199] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/087ff41f2a8c44e09389a5dd862ebbc4
I20260926 19:59:37.862994 15261 ts_tablet_manager.cc:1950] T 087ff41f2a8c44e09389a5dd862ebbc4 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:37.863332 15254 tablet_bootstrap.cc:492] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:37.863730 15254 ts_tablet_manager.cc:1403] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.000s
I20260926 19:59:37.866127 15254 raft_consensus.cc:359] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.866735 15254 raft_consensus.cc:385] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.866974 15254 raft_consensus.cc:740] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.870594 14540 master_service.cc:960] ReplaceTablet: received request to replace tablet 21847f1f910b458fad550d9ef426173e from {username='slave'} at 127.0.0.1:53020
I20260926 19:59:37.872057 14534 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet 087ff41f2a8c44e09389a5dd862ebbc4 (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
@ 0x7fe3fdab96db start_thread at ??:0
I20260926 19:59:37.876443 15255 tablet_bootstrap.cc:492] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:37.877452 14534 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet 087ff41f2a8c44e09389a5dd862ebbc4 (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:37.881783 15254 consensus_queue.cc:260] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.884032 15254 ts_tablet_manager.cc:1434] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
@ 0x7fe3f8e0461f clone at ??:0
I20260926 19:59:37.896723 15260 ts_tablet_manager.cc:1929] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:37.897156 15260 log.cc:1199] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/be4749d613b341c8879cd827e9ef27cc
I20260926 19:59:37.897958 15260 ts_tablet_manager.cc:1950] T be4749d613b341c8879cd827e9ef27cc P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
I20260926 19:59:37.899021 15255 tablet_bootstrap.cc:654] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:37.901437 15252 tablet_bootstrap.cc:492] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:37.902069 15252 ts_tablet_manager.cc:1403] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.065s user 0.013s sys 0.000s
I20260926 19:59:37.905279 15252 raft_consensus.cc:359] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.906149 15252 raft_consensus.cc:385] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.906505 15252 raft_consensus.cc:740] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.909471 15252 consensus_queue.cc:260] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.915405 15252 ts_tablet_manager.cc:1434] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.013s user 0.005s sys 0.002s
I20260926 19:59:37.918457 15255 tablet_bootstrap.cc:492] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:37.918874 15255 ts_tablet_manager.cc:1403] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.013s
I20260926 19:59:37.921476 15255 raft_consensus.cc:359] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.925567 15255 raft_consensus.cc:385] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:37.925896 15255 raft_consensus.cc:740] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:37.926783 15255 consensus_queue.cc:260] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:37.939774 15255 ts_tablet_manager.cc:1434] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.021s user 0.000s sys 0.006s
I20260926 19:59:37.953994 15267 raft_consensus.cc:493] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:37.954530 15267 raft_consensus.cc:515] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:37.956574 15267 leader_election.cc:290] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:37.980911 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8337dd2c7440c08ca282be7496806e" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" is_pre_election: true
I20260926 19:59:37.981567 15004 raft_consensus.cc:2468] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:37.983176 14793 leader_election.cc:304] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:37.985900 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8337dd2c7440c08ca282be7496806e" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" is_pre_election: true
I20260926 19:59:37.986501 15149 raft_consensus.cc:2468] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:37.990394 15267 raft_consensus.cc:2804] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:37.990928 15267 raft_consensus.cc:493] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:37.991259 15267 raft_consensus.cc:3060] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:37.999296 15267 raft_consensus.cc:515] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:38.003500 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8337dd2c7440c08ca282be7496806e" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
I20260926 19:59:38.004068 15004 raft_consensus.cc:3060] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:38.018465 15004 raft_consensus.cc:2468] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 1.
I20260926 19:59:38.019773 14793 leader_election.cc:304] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:38.026456 15310 raft_consensus.cc:2804] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:38.031553 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c8337dd2c7440c08ca282be7496806e" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
I20260926 19:59:38.032205 15149 raft_consensus.cc:3060] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:38.042469 15149 raft_consensus.cc:2468] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 1.
I20260926 19:59:38.042690 15310 raft_consensus.cc:697] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 1 LEADER]: Becoming Leader. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Running, Role: LEADER
I20260926 19:59:38.043797 15310 consensus_queue.cc:237] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:38.045074 15267 leader_election.cc:290] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Requested vote from peers 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:38.102149 15266 raft_consensus.cc:493] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:38.102654 15266 raft_consensus.cc:515] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:38.104578 15266 leader_election.cc:290] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:38.107403 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5da8e150c6844f9ba6e4a2cb2876fdec" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874" is_pre_election: true
I20260926 19:59:38.108280 14860 raft_consensus.cc:2468] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 0.
I20260926 19:59:38.109316 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5da8e150c6844f9ba6e4a2cb2876fdec" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" is_pre_election: true
I20260926 19:59:38.109939 15149 raft_consensus.cc:2468] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 0.
I20260926 19:59:38.113894 14940 leader_election.cc:304] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
W20260926 19:59:38.115099 15262 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:59:38.115866 15262 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260926 19:59:38.115778 15266 raft_consensus.cc:2804] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:38.116111 15266 raft_consensus.cc:493] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:38.116403 15266 raft_consensus.cc:3060] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Advancing to term 1
W20260926 19:59:38.116583 15262 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260926 19:59:38.123624 15266 raft_consensus.cc:515] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:38.127255 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5da8e150c6844f9ba6e4a2cb2876fdec" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874"
I20260926 19:59:38.127801 14860 raft_consensus.cc:3060] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:38.135165 14860 raft_consensus.cc:2468] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 1.
I20260926 19:59:38.136579 14940 leader_election.cc:304] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:38.143486 15266 leader_election.cc:290] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 election: Requested vote from peers 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:38.145359 15266 raft_consensus.cc:2804] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:38.146919 15266 raft_consensus.cc:697] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 LEADER]: Becoming Leader. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Running, Role: LEADER
I20260926 19:59:38.149667 15266 consensus_queue.cc:237] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:38.154125 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5da8e150c6844f9ba6e4a2cb2876fdec" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
I20260926 19:59:38.154749 15149 raft_consensus.cc:3060] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:38.165198 15149 raft_consensus.cc:2468] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 1.
W20260926 19:59:38.181424 15262 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 19:59:38.181825 15262 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 19:59:38.182202 15262 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:59:38.182572 15262 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:59:38.182943 15262 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:59:38.183298 15262 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:59:38.244016 15262 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:43737
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.254:32801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.233.254
--webserver_port=45057
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:59:38.245635 15262 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:59:38.247351 15262 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:59:38.259745 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:38.264983 15204 connection.cc:573] client connection to 127.11.233.253:39245 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:38.267542 15204 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.11.233.253:39245)
W20260926 19:59:38.268440 15204 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.11.233.253:39245)
W20260926 19:59:38.270520 15204 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.233.253:39245)
W20260926 19:59:38.277853 15204 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.11.233.253:39245)
W20260926 19:59:38.281572 14937 connection.cc:573] client connection to 127.11.233.253:39245 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:38.281535 15082 connection.cc:573] client connection to 127.11.233.253:39245 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:38.288859 14610 connection.cc:573] server connection from 127.0.0.1:60328 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:38.291323 14939 connection.cc:573] server connection from 127.0.0.1:43588 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:38.293094 15289 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 19:59:38.284219 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.11.233.253:39245 (local address 127.11.233.194:46071)
0926 19:59:38.285436 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0926 19:59:38.285486 (+ 50us) client_negotiation.cc:175] Beginning negotiation
0926 19:59:38.285769 (+ 283us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0926 19:59:38.292296 (+ 6527us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":381}
W20260926 19:59:38.294790 15327 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 19:59:38.280692 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.11.233.253:39245 (local address 127.0.0.1:53032)
0926 19:59:38.281801 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0926 19:59:38.281867 (+ 66us) client_negotiation.cc:175] Beginning negotiation
0926 19:59:38.282394 (+ 527us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0926 19:59:38.292484 (+ 10090us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":96}
W20260926 19:59:38.296159 15253 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 19:59:38.286371 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.11.233.253:39245 (local address 127.11.233.195:47821)
0926 19:59:38.287078 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0926 19:59:38.287118 (+ 40us) client_negotiation.cc:175] Beginning negotiation
0926 19:59:38.287378 (+ 260us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0926 19:59:38.294823 (+ 7445us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":124}
W20260926 19:59:38.300426 15251 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 19:59:38.290518 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.11.233.253:39245 (local address 127.11.233.193:58319)
0926 19:59:38.291261 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0926 19:59:38.291313 (+ 52us) client_negotiation.cc:175] Beginning negotiation
0926 19:59:38.291653 (+ 340us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0926 19:59:38.299083 (+ 7430us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":109}
I20260926 19:59:38.391050 15263 raft_consensus.cc:493] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:38.391711 15263 raft_consensus.cc:515] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:38.409360 15263 leader_election.cc:290] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755), 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:38.468703 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21847f1f910b458fad550d9ef426173e" candidate_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" is_pre_election: true
I20260926 19:59:38.469897 15004 raft_consensus.cc:2468] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3e2a17ea4c94765be709a7ec8af4c8a in term 0.
I20260926 19:59:38.471443 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21847f1f910b458fad550d9ef426173e" candidate_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874" is_pre_election: true
I20260926 19:59:38.471767 15083 leader_election.cc:304] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, e3e2a17ea4c94765be709a7ec8af4c8a; no voters:
I20260926 19:59:38.472275 14860 raft_consensus.cc:2468] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3e2a17ea4c94765be709a7ec8af4c8a in term 0.
I20260926 19:59:38.473040 15263 raft_consensus.cc:2804] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:38.473480 15263 raft_consensus.cc:493] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:38.473932 15263 raft_consensus.cc:3060] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:38.474999 15330 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fa6dae08245e46c98e01465e84732c92)
I20260926 19:59:38.475739 15330 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:38.484069 15263 raft_consensus.cc:515] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:38.494653 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21847f1f910b458fad550d9ef426173e" candidate_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874"
I20260926 19:59:38.495585 14860 raft_consensus.cc:3060] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:38.502785 15263 leader_election.cc:290] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [CANDIDATE]: Term 1 election: Requested vote from peers 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755), 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:38.503543 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21847f1f910b458fad550d9ef426173e" candidate_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
I20260926 19:59:38.504369 15004 raft_consensus.cc:3060] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:38.514526 15004 raft_consensus.cc:2468] T 21847f1f910b458fad550d9ef426173e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3e2a17ea4c94765be709a7ec8af4c8a in term 1.
I20260926 19:59:38.516415 15083 leader_election.cc:304] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, e3e2a17ea4c94765be709a7ec8af4c8a; no voters:
I20260926 19:59:38.517589 15263 raft_consensus.cc:2804] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:38.534097 15330 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:38.509456 14860 raft_consensus.cc:2468] T 21847f1f910b458fad550d9ef426173e P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3e2a17ea4c94765be709a7ec8af4c8a in term 1.
I20260926 19:59:38.536731 15263 raft_consensus.cc:697] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 LEADER]: Becoming Leader. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Running, Role: LEADER
I20260926 19:59:38.538091 15263 consensus_queue.cc:237] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
W20260926 19:59:38.574461 14611 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:38.588210 14608 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:38.589895 14608 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:38.591004 15330 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 19:59:39.018855 15340 consensus_queue.cc:1048] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:59:39.124549 15340 consensus_queue.cc:1048] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260926 19:59:39.133257 15341 consensus_queue.cc:1048] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:59:39.210189 15357 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fa6dae08245e46c98e01465e84732c92)
I20260926 19:59:39.210691 15357 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:39.228809 15357 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:39.230211 15354 consensus_queue.cc:1048] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:59:39.273407 14608 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:39.276790 14611 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 19:59:39.277431 14611 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:39.279251 15357 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260926 19:59:39.503556 15349 consensus_queue.cc:1048] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:39.583142 15349 consensus_queue.cc:1048] T 21847f1f910b458fad550d9ef426173e P e3e2a17ea4c94765be709a7ec8af4c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:59:40.155028 15374 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fa6dae08245e46c98e01465e84732c92)
I20260926 19:59:40.156311 15374 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:40.201864 15374 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
W20260926 19:59:40.221341 14608 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:40.224215 14611 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 19:59:40.224812 14611 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb0f126418d54df489a4ce95a765b8d3; no voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:40.225615 15374 raft_consensus.cc:2749] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260926 19:59:39.664474 15321 debug-util.cc:398] Leaking SignalData structure 0x7b08000283c0 after lost signal to thread 15262
W20260926 19:59:38.260843 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:40.235388 15262 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.973s user 0.002s sys 0.000s
W20260926 19:59:40.237375 15262 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.976s user 0.002s sys 0.000s
W20260926 19:59:40.238646 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:40.238765 15324 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1974 milliseconds
I20260926 19:59:40.238700 15262 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:59:40.242146 15262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:59:40.244510 15262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:59:40.246060 15262 hybrid_clock.cc:648] HybridClock initialized: now 1790452780246010 us; error 48 us; skew 500 ppm
I20260926 19:59:40.246899 15262 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:59:40.252914 15262 webserver.cc:533] Webserver started at http://127.11.233.254:45057/ using document root <none> and password file <none>
I20260926 19:59:40.253994 15262 fs_manager.cc:362] Metadata directory not provided
I20260926 19:59:40.254313 15262 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:59:40.262096 15262 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 19:59:40.266778 15385 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:40.267859 15262 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 19:59:40.268216 15262 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
uuid: "26c8ef2c095f418083f33f8f0a734c3a"
format_stamp: "Formatted at 2026-09-26 19:59:21 on dist-test-slave-jcdj"
I20260926 19:59:40.270198 15262 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:59:40.334472 15262 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:59:40.336026 15262 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:59:40.336560 15262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:59:40.419240 15262 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.254:32801
I20260926 19:59:40.419333 15437 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.254:32801 every 8 connection(s)
I20260926 19:59:40.422632 15262 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-0/data/info.pb
I20260926 19:59:40.427551 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 15262
I20260926 19:59:40.437739 15438 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:59:40.445232 15438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Bootstrap starting.
I20260926 19:59:40.460639 15438 log.cc:826] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Log is configured to *not* fsync() on all Append() calls
I20260926 19:59:40.478940 15438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:59:40.479691 15438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Bootstrap complete.
I20260926 19:59:40.497762 15438 raft_consensus.cc:359] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:40.500010 15438 raft_consensus.cc:740] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c8ef2c095f418083f33f8f0a734c3a, State: Initialized, Role: FOLLOWER
I20260926 19:59:40.500835 15438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:40.503588 15442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:40.504410 15438 sys_catalog.cc:565] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:59:40.504408 15442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:40.538975 15456 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:40.539479 15456 catalog_manager.cc:1601] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: loading cluster ID for follower catalog manager: success
I20260926 19:59:40.547168 15456 catalog_manager.cc:1623] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: acquiring CA information for follower catalog manager: success
I20260926 19:59:40.554841 15456 catalog_manager.cc:1651] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:59:40.564869 15262 master_runner.cc:187] Error getting master registration for 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111),
I20260926 19:59:40.599489 15262 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:59:40.611325 15196 heartbeater.cc:344] Connected to a master server at 127.11.233.254:32801
I20260926 19:59:40.611833 15196 heartbeater.cc:499] Master 127.11.233.254:32801 was elected leader, sending a full tablet report...
I20260926 19:59:40.620267 15401 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" instance_seqno: 1790452775589282) as {username='slave'} at 127.11.233.195:60715; Asking this server to re-register.
I20260926 19:59:40.627306 15196 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:40.633241 15401 ts_manager.cc:194] Registered new tserver with Master: e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:40.671705 14909 heartbeater.cc:344] Connected to a master server at 127.11.233.254:32801
I20260926 19:59:40.676216 15401 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" instance_seqno: 1790452770962163) as {username='slave'} at 127.11.233.193:48339; Asking this server to re-register.
I20260926 19:59:40.677604 14909 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:40.681248 15401 ts_manager.cc:194] Registered new tserver with Master: 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755)
I20260926 19:59:40.738667 15052 heartbeater.cc:344] Connected to a master server at 127.11.233.254:32801
I20260926 19:59:40.743755 15401 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" instance_seqno: 1790452773079792) as {username='slave'} at 127.11.233.194:57369; Asking this server to re-register.
I20260926 19:59:40.745077 15052 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:40.748318 15401 ts_manager.cc:194] Registered new tserver with Master: 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:40.975754 15442 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:40.976248 15442 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:40.979018 15442 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:40.982434 15386 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:40.996248 15386 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:41.001708 14633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "eb0f126418d54df489a4ce95a765b8d3" is_pre_election: true
I20260926 19:59:41.002329 14633 raft_consensus.cc:2393] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 26c8ef2c095f418083f33f8f0a734c3a in current term 4: Already voted for candidate fa6dae08245e46c98e01465e84732c92 in this term.
I20260926 19:59:41.003535 15389 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a; no voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:41.004289 15442 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 3 FOLLOWER]: Advancing to term 4
I20260926 19:59:41.011124 15442 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260926 19:59:41.070061 15468 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader fa6dae08245e46c98e01465e84732c92)
I20260926 19:59:41.070464 15468 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:41.072291 15468 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
W20260926 19:59:41.076702 14608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260926 19:59:41.081900 14608 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:41.083334 15412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 5 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a" is_pre_election: true
I20260926 19:59:41.084018 15412 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb0f126418d54df489a4ce95a765b8d3 in term 4.
I20260926 19:59:41.085000 14611 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3; no voters: fa6dae08245e46c98e01465e84732c92
I20260926 19:59:41.085681 15468 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260926 19:59:41.086033 15468 raft_consensus.cc:493] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader fa6dae08245e46c98e01465e84732c92)
I20260926 19:59:41.086369 15468 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 4 FOLLOWER]: Advancing to term 5
I20260926 19:59:41.093828 15468 raft_consensus.cc:515] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:41.095976 15468 leader_election.cc:290] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 election: Requested vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245)
I20260926 19:59:41.096652 15412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb0f126418d54df489a4ce95a765b8d3" candidate_term: 5 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "26c8ef2c095f418083f33f8f0a734c3a"
I20260926 19:59:41.097206 15412 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 4 FOLLOWER]: Advancing to term 5
W20260926 19:59:41.100180 14608 leader_election.cc:336] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
I20260926 19:59:41.104435 15412 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb0f126418d54df489a4ce95a765b8d3 in term 5.
I20260926 19:59:41.107682 14611 leader_election.cc:304] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3; no voters: fa6dae08245e46c98e01465e84732c92
I20260926 19:59:41.108682 15468 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 5 FOLLOWER]: Leader election won for term 5
I20260926 19:59:41.111853 15468 raft_consensus.cc:697] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [term 5 LEADER]: Becoming Leader. State: Replica: eb0f126418d54df489a4ce95a765b8d3, State: Running, Role: LEADER
I20260926 19:59:41.112920 15468 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 4.24, Last appended by leader: 24, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:41.118054 15472 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0f126418d54df489a4ce95a765b8d3. Latest consensus state: current_term: 5 leader_uuid: "eb0f126418d54df489a4ce95a765b8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:41.118851 15472 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:41.702885 15412 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Refusing update from remote peer eb0f126418d54df489a4ce95a765b8d3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 25. (index mismatch)
I20260926 19:59:41.704514 15480 consensus_queue.cc:1048] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 19:59:41.714708 14608 consensus_peers.cc:597] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 -> Peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Couldn't send request to peer fa6dae08245e46c98e01465e84732c92. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 19:59:41.739491 15484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb0f126418d54df489a4ce95a765b8d3. Latest consensus state: current_term: 5 leader_uuid: "eb0f126418d54df489a4ce95a765b8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:41.740199 15484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:41.744880 15487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "eb0f126418d54df489a4ce95a765b8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:41.745452 15484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "eb0f126418d54df489a4ce95a765b8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:41.748323 15484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:41.748919 15487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:41.765929 15480 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "eb0f126418d54df489a4ce95a765b8d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:41.766800 15480 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3 [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:41.773533 15474 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 19:59:41.782879 15474 catalog_manager.cc:692] Loaded metadata for table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]
I20260926 19:59:41.784713 15474 catalog_manager.cc:692] Loaded metadata for table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]
I20260926 19:59:41.789858 15474 catalog_manager.cc:692] Loaded metadata for table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]
I20260926 19:59:41.797716 15474 tablet_loader.cc:96] loaded metadata for tablet 0c8337dd2c7440c08ca282be7496806e (table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792])
I20260926 19:59:41.800760 15474 tablet_loader.cc:96] loaded metadata for tablet 5da8e150c6844f9ba6e4a2cb2876fdec (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb])
I20260926 19:59:41.802002 15474 tablet_loader.cc:96] loaded metadata for tablet 65265ecba84e438ebe77c4de229cedb5 (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875])
I20260926 19:59:41.804232 15474 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 19:59:41.804687 15485 mvcc.cc:204] Tried to move back new op lower bound from 7333694578177204224 to 7333694570807037952. Current Snapshot: MvccSnapshot[applied={T|T < 7333694578177204224}]
I20260926 19:59:41.808487 15474 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:41.808709 15474 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:59:41.814162 15474 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:59:41.818029 15474 catalog_manager.cc:6230] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: Loaded TSK: 0
I20260926 19:59:41.819221 15474 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:59:41.822103 15474 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 19:59:41.824941 15474 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260926 19:59:41.868640 15129 tablet_service.cc:1511] Processing CreateTablet for tablet 65265ecba84e438ebe77c4de229cedb5 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:41.869880 15129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65265ecba84e438ebe77c4de229cedb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:41.887732 14984 tablet_service.cc:1511] Processing CreateTablet for tablet 65265ecba84e438ebe77c4de229cedb5 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:41.889313 14984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65265ecba84e438ebe77c4de229cedb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:41.895465 14623 master_service.cc:960] ReplaceTablet: received request to replace tablet 5da8e150c6844f9ba6e4a2cb2876fdec from {username='slave'} at 127.0.0.1:43236
I20260926 19:59:41.899189 14624 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43236:
table { table_name: "default.Table_e144b6535e8448a3b04ecd97e88b06a9" } modify_external_catalogs: true
I20260926 19:59:41.900007 14624 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43236:
table { table_name: "default.Table_e144b6535e8448a3b04ecd97e88b06a9" } modify_external_catalogs: true
I20260926 19:59:41.904222 14622 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43236:
name: "default.Table_c74a17cf08a9482ab2915e2d7ff9fbeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:41.906584 14621 master_service.cc:960] ReplaceTablet: received request to replace tablet 21847f1f910b458fad550d9ef426173e from {username='slave'} at 127.0.0.1:43236
F20260926 19:59:41.911049 14701 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1790452776, eventType=CREATE_TABLE, dbName=default, tableName=table_131b6490764e47dc8a7a589300bbf272, message={"server":"","servicePrincipal":"","db":"default","table":"table_131b6490764e47dc8a7a589300bbf272","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_131b6490764e47dc8a7a589300bbf272\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790452775},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/warehouse/table_131b6490764e47dc8a7a589300bbf272\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_131b6490764e47dc8a7a589300bbf272\",\"kudu.table_id\":\"53eb5bec635046daa62c8b441e0d404c\",\"transient_lastDdlTime\":\"1790452775\",\"comment\":\"\",\"kudu.master_addresses\":\"127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"b4293131cc434d2194a1cdf2d39da0b5\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790452776,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1790452776, eventType=CREATE_TABLE, dbName=default, tableName=table_214fa130d427469eaf37415c0767c7e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_214fa130d427469eaf37415c0767c7e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_214fa130d427469eaf37415c0767c7e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790452776},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/warehouse/table_214fa130d427469eaf37415c0767c7e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_214fa130d427469eaf37415c0767c7e3\",\"kudu.table_id\":\"a07796662b5d4b0b8b019aa4e111b875\",\"transient_lastDdlTime\":\"1790452776\",\"comment\":\"\",\"kudu.master_addresses\":\"127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"b4293131cc434d2194a1cdf2d39da0b5\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790452776,"files":[]}, messageFormat=json-0.2, catName=hive);
I20260926 19:59:41.912317 14839 tablet_service.cc:1511] Processing CreateTablet for tablet 65265ecba84e438ebe77c4de229cedb5 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
*** Check failure stack trace: ***
I20260926 19:59:41.913866 14839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65265ecba84e438ebe77c4de229cedb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:41.918776 15501 tablet_bootstrap.cc:492] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:41.918531 14621 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43236:
table { table_name: "default.Table_7a1c4598445a4ac5845670a68237cb86" } new_table_name: "default.Table_2d6e52c75c4f4413a0185f462c80af99" modify_external_catalogs: true
I20260926 19:59:41.925628 14620 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43236:
name: "default.Table_048abbb957574bd9904e0ba925421cee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:41.930136 15501 tablet_bootstrap.cc:654] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:41.931123 14619 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43236:
name: "default.Table_3c580a21625b41a4894dd8072db5ba09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:41.938784 14618 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43236:
name: "default.Table_2369b73124a04219b46431eeadde2ac7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:41.945024 14617 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43236:
table { table_name: "default.Table_214fa130d427469eaf37415c0767c7e3" } new_table_name: "default.Table_110ad590096942f0a032e632ccedc05f" modify_external_catalogs: true
I20260926 19:59:41.946699 14623 catalog_manager.cc:6133] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: Sending DeleteTablet for 3 replicas of tablet 5da8e150c6844f9ba6e4a2cb2876fdec
I20260926 19:59:41.949313 14840 tablet_service.cc:1558] Processing DeleteTablet for tablet 5da8e150c6844f9ba6e4a2cb2876fdec with delete_type TABLET_DATA_DELETED (replaced by tablet 197039cbece04593bc7b11041dbf1c8e) from {username='slave'} at 127.0.0.1:60570
I20260926 19:59:41.951383 14982 tablet_service.cc:1558] Processing DeleteTablet for tablet 5da8e150c6844f9ba6e4a2cb2876fdec with delete_type TABLET_DATA_DELETED (replaced by tablet 197039cbece04593bc7b11041dbf1c8e) from {username='slave'} at 127.0.0.1:43604
I20260926 19:59:41.952481 15129 tablet_service.cc:1558] Processing DeleteTablet for tablet 5da8e150c6844f9ba6e4a2cb2876fdec with delete_type TABLET_DATA_DELETED (replaced by tablet 197039cbece04593bc7b11041dbf1c8e) from {username='slave'} at 127.0.0.1:38688
I20260926 19:59:41.954506 15500 tablet_bootstrap.cc:492] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:41.955974 14623 catalog_manager.cc:6934] ReplaceTablet: tablet 5da8e150c6844f9ba6e4a2cb2876fdec deleted and replaced by tablet 197039cbece04593bc7b11041dbf1c8e
I20260926 19:59:41.959098 15502 tablet_bootstrap.cc:492] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:41.966189 15504 tablet_replica.cc:333] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:41.967242 15504 raft_consensus.cc:2243] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:41.970649 15504 raft_consensus.cc:2272] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:41.974077 15504 ts_tablet_manager.cc:1916] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:41.976727 14983 tablet_service.cc:1511] Processing CreateTablet for tablet 197039cbece04593bc7b11041dbf1c8e (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:41.977308 15128 tablet_service.cc:1511] Processing CreateTablet for tablet 197039cbece04593bc7b11041dbf1c8e (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:41.978235 14983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197039cbece04593bc7b11041dbf1c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:41.978490 15128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197039cbece04593bc7b11041dbf1c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:41.978771 15500 tablet_bootstrap.cc:654] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:41.986496 15500 tablet_bootstrap.cc:492] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:41.987013 15500 ts_tablet_manager.cc:1403] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.004s
I20260926 19:59:41.987288 15506 tablet_replica.cc:333] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:41.988097 15506 raft_consensus.cc:2243] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:41.988615 15506 raft_consensus.cc:2272] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:41.990583 15506 ts_tablet_manager.cc:1916] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:41.990460 15500 raft_consensus.cc:359] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:41.991582 15500 raft_consensus.cc:385] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:41.991935 15500 raft_consensus.cc:740] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:41.992828 15500 consensus_queue.cc:260] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:41.994756 15505 tablet_replica.cc:333] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:41.975979 14840 tablet_service.cc:1511] Processing CreateTablet for tablet 197039cbece04593bc7b11041dbf1c8e (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:41.995679 15505 raft_consensus.cc:2243] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:59:41.996382 15501 tablet_bootstrap.cc:492] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:41.996565 14840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197039cbece04593bc7b11041dbf1c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:41.996810 15501 ts_tablet_manager.cc:1403] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.078s user 0.009s sys 0.003s
I20260926 19:59:41.996898 15505 raft_consensus.cc:2272] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:41.999104 15505 ts_tablet_manager.cc:1916] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:42.002182 15501 raft_consensus.cc:359] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:42.002856 15501 raft_consensus.cc:385] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:42.003122 15501 raft_consensus.cc:740] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:42.003791 15501 consensus_queue.cc:260] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:41.994590 15502 tablet_bootstrap.cc:654] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
@ 0x7fefa1462920 google::LogMessage::Flush() at ??:0
I20260926 19:59:42.024163 15500 ts_tablet_manager.cc:1434] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.037s user 0.005s sys 0.001s
I20260926 19:59:42.029937 15051 heartbeater.cc:499] Master 127.11.233.252:37917 was elected leader, sending a full tablet report...
W20260926 19:59:42.038833 14623 catalog_manager.cc:5671] Ignoring report from unknown tablet 5da8e150c6844f9ba6e4a2cb2876fdec
W20260926 19:59:42.039096 14623 catalog_manager.cc:5671] Ignoring report from unknown tablet 21847f1f910b458fad550d9ef426173e
I20260926 19:59:42.042003 15505 ts_tablet_manager.cc:1929] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:42.040865 14623 catalog_manager.cc:5838] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193). New cstate: current_term: 1 leader_uuid: "56196546dad349b9bd344c4e0b6a6874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } }
I20260926 19:59:42.042402 15505 log.cc:1199] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/5da8e150c6844f9ba6e4a2cb2876fdec
I20260926 19:59:42.043409 15505 ts_tablet_manager.cc:1950] T 5da8e150c6844f9ba6e4a2cb2876fdec P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
I20260926 19:59:42.044857 15500 tablet_bootstrap.cc:492] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:42.020840 15506 ts_tablet_manager.cc:1929] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:42.045642 15506 log.cc:1199] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/5da8e150c6844f9ba6e4a2cb2876fdec
I20260926 19:59:42.046435 15506 ts_tablet_manager.cc:1950] T 5da8e150c6844f9ba6e4a2cb2876fdec P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:42.036072 15501 ts_tablet_manager.cc:1434] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.036s user 0.006s sys 0.001s
I20260926 19:59:42.049238 14611 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet 5da8e150c6844f9ba6e4a2cb2876fdec (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:42.051007 15501 tablet_bootstrap.cc:492] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
@ 0x7fefa146268f google::LogMessage::~LogMessage() at ??:0
I20260926 19:59:42.062906 15501 tablet_bootstrap.cc:654] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:42.065811 15502 tablet_bootstrap.cc:492] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:42.066272 15502 ts_tablet_manager.cc:1403] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.107s user 0.011s sys 0.007s
I20260926 19:59:42.067821 15500 tablet_bootstrap.cc:654] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:42.069844 14609 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet 5da8e150c6844f9ba6e4a2cb2876fdec (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:42.072324 15502 raft_consensus.cc:359] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:42.072913 15195 heartbeater.cc:499] Master 127.11.233.252:37917 was elected leader, sending a full tablet report...
I20260926 19:59:42.073040 15502 raft_consensus.cc:385] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:42.073338 15502 raft_consensus.cc:740] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
W20260926 19:59:42.078395 14616 catalog_manager.cc:5671] Ignoring report from unknown tablet 21847f1f910b458fad550d9ef426173e
I20260926 19:59:42.081875 15502 consensus_queue.cc:260] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } }
I20260926 19:59:42.084704 15500 tablet_bootstrap.cc:492] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:42.085143 15500 ts_tablet_manager.cc:1403] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.006s
I20260926 19:59:42.085364 15504 ts_tablet_manager.cc:1929] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:42.085786 15504 log.cc:1199] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/5da8e150c6844f9ba6e4a2cb2876fdec
I20260926 19:59:42.086709 15504 ts_tablet_manager.cc:1950] T 5da8e150c6844f9ba6e4a2cb2876fdec P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:42.090068 15501 tablet_bootstrap.cc:492] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:42.090494 15501 ts_tablet_manager.cc:1403] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.000s
I20260926 19:59:42.093221 15501 raft_consensus.cc:359] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.094147 15501 raft_consensus.cc:385] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:42.094458 15501 raft_consensus.cc:740] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:42.095059 15502 ts_tablet_manager.cc:1434] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.028s user 0.006s sys 0.002s
I20260926 19:59:42.095302 15501 consensus_queue.cc:260] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.097805 15501 ts_tablet_manager.cc:1434] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 19:59:42.099040 15500 raft_consensus.cc:359] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.099707 15500 raft_consensus.cc:385] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:42.100003 15500 raft_consensus.cc:740] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:42.102430 15502 tablet_bootstrap.cc:492] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:42.107170 15500 consensus_queue.cc:260] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.115724 15500 ts_tablet_manager.cc:1434] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.030s user 0.002s sys 0.005s
@ 0x7fefac5f535c kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 19:59:42.127715 14906 heartbeater.cc:499] Master 127.11.233.252:37917 was elected leader, sending a full tablet report...
W20260926 19:59:42.133878 14615 catalog_manager.cc:5671] Ignoring report from unknown tablet 21847f1f910b458fad550d9ef426173e
I20260926 19:59:42.134892 14611 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet 5da8e150c6844f9ba6e4a2cb2876fdec (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:42.140846 14623 master_service.cc:960] ReplaceTablet: received request to replace tablet 65265ecba84e438ebe77c4de229cedb5 from {username='slave'} at 127.0.0.1:43236
I20260926 19:59:42.147526 15502 tablet_bootstrap.cc:654] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
@ 0x7fefac5f4b1c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 19:59:42.159510 14623 catalog_manager.cc:6133] T 00000000000000000000000000000000 P eb0f126418d54df489a4ce95a765b8d3: Sending DeleteTablet for 3 replicas of tablet 65265ecba84e438ebe77c4de229cedb5
I20260926 19:59:42.161935 14983 tablet_service.cc:1558] Processing DeleteTablet for tablet 65265ecba84e438ebe77c4de229cedb5 with delete_type TABLET_DATA_DELETED (replaced by tablet 7171f772b30648afba52e793d0d0228b) from {username='slave'} at 127.0.0.1:43604
I20260926 19:59:42.162485 14623 catalog_manager.cc:6934] ReplaceTablet: tablet 65265ecba84e438ebe77c4de229cedb5 deleted and replaced by tablet 7171f772b30648afba52e793d0d0228b
I20260926 19:59:42.165820 15505 tablet_replica.cc:333] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:42.166579 15505 raft_consensus.cc:2243] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:42.167007 14623 master_service.cc:960] ReplaceTablet: received request to replace tablet 21847f1f910b458fad550d9ef426173e from {username='slave'} at 127.0.0.1:43236
I20260926 19:59:42.167044 15505 raft_consensus.cc:2272] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:42.169118 15128 tablet_service.cc:1558] Processing DeleteTablet for tablet 65265ecba84e438ebe77c4de229cedb5 with delete_type TABLET_DATA_DELETED (replaced by tablet 7171f772b30648afba52e793d0d0228b) from {username='slave'} at 127.0.0.1:38688
I20260926 19:59:42.169581 15505 ts_tablet_manager.cc:1916] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:42.169979 15506 tablet_replica.cc:333] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:42.170719 15506 raft_consensus.cc:2243] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:42.171164 15506 raft_consensus.cc:2272] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:42.173285 15506 ts_tablet_manager.cc:1916] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:42.179395 14840 tablet_service.cc:1558] Processing DeleteTablet for tablet 65265ecba84e438ebe77c4de229cedb5 with delete_type TABLET_DATA_DELETED (replaced by tablet 7171f772b30648afba52e793d0d0228b) from {username='slave'} at 127.0.0.1:60570
I20260926 19:59:42.180119 15504 tablet_replica.cc:333] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:42.180840 15504 raft_consensus.cc:2243] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:42.181335 15504 raft_consensus.cc:2272] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:42.183586 15504 ts_tablet_manager.cc:1916] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:42.198701 15505 ts_tablet_manager.cc:1929] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:42.199115 15505 log.cc:1199] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/65265ecba84e438ebe77c4de229cedb5
I20260926 19:59:42.199800 15505 ts_tablet_manager.cc:1950] T 65265ecba84e438ebe77c4de229cedb5 P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
I20260926 19:59:42.202032 15504 ts_tablet_manager.cc:1929] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:42.202440 15504 log.cc:1199] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/65265ecba84e438ebe77c4de229cedb5
I20260926 19:59:42.202744 14609 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet 65265ecba84e438ebe77c4de229cedb5 (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:42.203130 15504 ts_tablet_manager.cc:1950] T 65265ecba84e438ebe77c4de229cedb5 P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
@ 0x7fefac5f8b92 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260926 19:59:42.210260 14611 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet 65265ecba84e438ebe77c4de229cedb5 (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:42.214675 14983 tablet_service.cc:1511] Processing CreateTablet for tablet 7171f772b30648afba52e793d0d0228b (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:42.215071 14840 tablet_service.cc:1511] Processing CreateTablet for tablet 7171f772b30648afba52e793d0d0228b (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:42.215830 14983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7171f772b30648afba52e793d0d0228b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:42.216260 14840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7171f772b30648afba52e793d0d0228b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:42.223694 15128 tablet_service.cc:1511] Processing CreateTablet for tablet 7171f772b30648afba52e793d0d0228b (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:42.227517 15128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7171f772b30648afba52e793d0d0228b. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fefac5f8b4a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260926 19:59:42.244859 15500 tablet_bootstrap.cc:492] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:42.247236 15506 ts_tablet_manager.cc:1929] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:42.247653 15506 log.cc:1199] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/65265ecba84e438ebe77c4de229cedb5
I20260926 19:59:42.248402 15506 ts_tablet_manager.cc:1950] T 65265ecba84e438ebe77c4de229cedb5 P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
@ 0x7fefac5f8ada std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260926 19:59:42.256803 15500 tablet_bootstrap.cc:654] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:42.261236 15501 tablet_bootstrap.cc:492] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:42.262764 14611 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet 65265ecba84e438ebe77c4de229cedb5 (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]) successfully deleted
I20260926 19:59:42.267033 15502 tablet_bootstrap.cc:492] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:42.267494 15502 ts_tablet_manager.cc:1403] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.172s user 0.036s sys 0.013s
I20260926 19:59:42.272792 15501 tablet_bootstrap.cc:654] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:42.274454 15502 raft_consensus.cc:359] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.275197 15502 raft_consensus.cc:385] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:42.275520 15502 raft_consensus.cc:740] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
@ 0x7fefac5f8aa2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260926 19:59:42.276259 15502 consensus_queue.cc:260] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.282882 15502 ts_tablet_manager.cc:1434] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260926 19:59:42.288203 15502 tablet_bootstrap.cc:492] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:42.297266 15500 tablet_bootstrap.cc:492] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:42.297267 15501 tablet_bootstrap.cc:492] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:42.297756 15500 ts_tablet_manager.cc:1403] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.053s user 0.013s sys 0.008s
I20260926 19:59:42.297763 15501 ts_tablet_manager.cc:1403] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.003s
I20260926 19:59:42.300436 15500 raft_consensus.cc:359] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.301097 15500 raft_consensus.cc:385] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:42.301782 15501 raft_consensus.cc:359] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.302495 15501 raft_consensus.cc:385] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:42.301359 15500 raft_consensus.cc:740] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:42.302781 15501 raft_consensus.cc:740] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:42.303323 15500 consensus_queue.cc:260] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
@ 0x7fefac5f7d9e std::__1::__function::__func<>::operator()() at ??:0
I20260926 19:59:42.306228 15502 tablet_bootstrap.cc:654] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
@ 0x5bd4e5 std::__1::__function::__value_func<>::operator()() at ??:?
I20260926 19:59:42.309092 15501 consensus_queue.cc:260] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
@ 0x5bd31a std::__1::function<>::operator()() at ??:?
I20260926 19:59:42.315145 15501 ts_tablet_manager.cc:1434] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260926 19:59:42.322108 15502 tablet_bootstrap.cc:492] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:42.322580 15502 ts_tablet_manager.cc:1403] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.035s user 0.011s sys 0.000s
I20260926 19:59:42.325441 15502 raft_consensus.cc:359] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.326331 15502 raft_consensus.cc:385] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:42.326618 15502 raft_consensus.cc:740] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:42.327471 15502 consensus_queue.cc:260] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.328199 15500 ts_tablet_manager.cc:1434] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260926 19:59:42.335558 15502 ts_tablet_manager.cc:1434] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
@ 0x7fefa365881b kudu::Thread::SuperviseThread() at ??:0
@ 0x4569a0 __tsan_thread_start_func at ??:?
I20260926 19:59:42.367372 15478 raft_consensus.cc:493] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:42.367854 15478 raft_consensus.cc:515] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
@ 0x7fefa4db96db start_thread at ??:0
I20260926 19:59:42.370349 15478 leader_election.cc:290] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:42.371840 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7171f772b30648afba52e793d0d0228b" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" is_pre_election: true
I20260926 19:59:42.372546 15004 raft_consensus.cc:2468] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:42.373494 14623 master_service.cc:960] ReplaceTablet: received request to replace tablet 21847f1f910b458fad550d9ef426173e from {username='slave'} at 127.0.0.1:43236
I20260926 19:59:42.374151 14793 leader_election.cc:304] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:42.375523 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7171f772b30648afba52e793d0d0228b" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" is_pre_election: true
I20260926 19:59:42.376116 15149 raft_consensus.cc:2468] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:42.377530 15478 raft_consensus.cc:2804] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:42.377888 15478 raft_consensus.cc:493] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:42.378203 15478 raft_consensus.cc:3060] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:42.384671 14623 master_service.cc:960] ReplaceTablet: received request to replace tablet 197039cbece04593bc7b11041dbf1c8e from {username='slave'} at 127.0.0.1:43236
@ 0x7fefa010461f clone at ??:0
I20260926 19:59:42.392112 15478 raft_consensus.cc:515] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.395303 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7171f772b30648afba52e793d0d0228b" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
I20260926 19:59:42.395790 15478 leader_election.cc:290] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Requested vote from peers 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:42.395848 15004 raft_consensus.cc:3060] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:42.396919 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7171f772b30648afba52e793d0d0228b" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
I20260926 19:59:42.397460 15149 raft_consensus.cc:3060] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:42.404178 15464 raft_consensus.cc:493] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:42.404819 15464 raft_consensus.cc:515] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.405606 15004 raft_consensus.cc:2468] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 1.
I20260926 19:59:42.407097 14793 leader_election.cc:304] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:42.407943 15464 leader_election.cc:290] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:42.408605 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197039cbece04593bc7b11041dbf1c8e" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874" is_pre_election: true
I20260926 19:59:42.409103 15148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197039cbece04593bc7b11041dbf1c8e" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" is_pre_election: true
I20260926 19:59:42.409214 14860 raft_consensus.cc:2468] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 0.
I20260926 19:59:42.409694 15148 raft_consensus.cc:2468] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 0.
I20260926 19:59:42.410636 14940 leader_election.cc:304] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:42.413093 15464 raft_consensus.cc:2804] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:42.413062 15478 raft_consensus.cc:2804] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:42.413084 15149 raft_consensus.cc:2468] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 1.
I20260926 19:59:42.413426 15464 raft_consensus.cc:493] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:42.413739 15464 raft_consensus.cc:3060] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:42.421084 15464 raft_consensus.cc:515] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.422987 15464 leader_election.cc:290] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 election: Requested vote from peers 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755), e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:42.424341 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197039cbece04593bc7b11041dbf1c8e" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
I20260926 19:59:42.424901 15149 raft_consensus.cc:3060] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:42.430439 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197039cbece04593bc7b11041dbf1c8e" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874"
I20260926 19:59:42.431007 14860 raft_consensus.cc:3060] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:42.435495 15149 raft_consensus.cc:2468] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 1.
I20260926 19:59:42.438114 14940 leader_election.cc:304] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, e3e2a17ea4c94765be709a7ec8af4c8a; no voters:
I20260926 19:59:42.438702 15478 raft_consensus.cc:697] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 1 LEADER]: Becoming Leader. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Running, Role: LEADER
I20260926 19:59:42.439007 15464 raft_consensus.cc:2804] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:42.445106 14860 raft_consensus.cc:2468] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 1.
I20260926 19:59:42.446033 15478 consensus_queue.cc:237] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
I20260926 19:59:42.453248 15464 raft_consensus.cc:697] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 LEADER]: Becoming Leader. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Running, Role: LEADER
I20260926 19:59:42.454208 15464 consensus_queue.cc:237] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } }
W20260926 19:59:42.636227 15207 connection.cc:573] client connection to 127.11.233.252:37917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:42.638366 14940 connection.cc:573] client connection to 127.11.233.252:37917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:42.639438 14795 connection.cc:573] client connection to 127.11.233.252:37917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:42.642290 15207 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.11.233.252:37917)
W20260926 19:59:42.642966 15207 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.233.252:37917)
W20260926 19:59:42.644518 15207 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.11.233.252:37917)
W20260926 19:59:42.648229 15207 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.11.233.252:37917)
I20260926 19:59:42.719133 15533 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb0f126418d54df489a4ce95a765b8d3)
I20260926 19:59:42.719622 15533 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:42.721730 15533 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:42.727326 15386 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:42.728595 15389 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:42.728953 15389 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a; no voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:42.729565 15533 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 19:59:43.303931 15542 consensus_queue.cc:1048] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:43.326447 15550 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb0f126418d54df489a4ce95a765b8d3)
I20260926 19:59:43.327126 15550 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:43.328187 15548 consensus_queue.cc:1048] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:43.335210 15550 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:43.349355 15386 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:43.350441 15389 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:43.350818 15389 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a; no voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:43.351436 15550 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260926 19:59:43.448904 15554 consensus_queue.cc:1048] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:43.462591 15560 consensus_queue.cc:1048] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:43.934934 12199 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
/tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.253:39245
--webserver_interface=127.11.233.253
--webserver_port=39383
--builtin_ntp_servers=127.11.233.212:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--hive_metastore_uris=thrift://127.0.0.1:43737
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260926 19:59:43.986220 15568 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb0f126418d54df489a4ce95a765b8d3)
I20260926 19:59:43.987021 15568 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:43.989306 15568 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:43.996502 15386 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:43.997056 15389 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:43.997475 15389 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a; no voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:43.998152 15568 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260926 19:59:44.300573 15565 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:59:44.301132 15565 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:59:44.301623 15565 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260926 19:59:44.337925 15565 flags.cc:432] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260926 19:59:44.338186 15565 flags.cc:432] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260926 19:59:44.338385 15565 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:59:44.338585 15565 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:59:44.338784 15565 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:59:44.338984 15565 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:59:44.380719 15565 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.233.212:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:43737
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.233.253:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.233.253
--webserver_port=39383
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3b18f0139b1d21a072b7353269301fba16dca54e
build type FASTDEBUG
built by None at 26 Sep 2026 19:43:25 UTC on e7f111948823
build id 14014
TSAN enabled
I20260926 19:59:44.382124 15565 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:59:44.383689 15565 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260926 19:59:44.394914 15583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:44.662346 15592 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb0f126418d54df489a4ce95a765b8d3)
I20260926 19:59:44.663009 15592 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:44.698998 15592 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:44.727314 15386 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245): Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111)
W20260926 19:59:44.728664 15389 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:44.729310 15389 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a; no voters: eb0f126418d54df489a4ce95a765b8d3, fa6dae08245e46c98e01465e84732c92
I20260926 19:59:44.735967 15592 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260926 19:59:45.462285 14793 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.194:34597, user_credentials={real_user=slave}} blocked reactor thread for 82240us
I20260926 19:59:45.620184 15262 master_runner.cc:187] Error getting master registration for 127.11.233.253:39245: Network error: Client connection negotiation failed: client connection to 127.11.233.253:39245: connect: Connection refused (error 111),
I20260926 19:59:45.620611 15262 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 19:59:45.799057 15582 debug-util.cc:398] Leaking SignalData structure 0x7b08000283c0 after lost signal to thread 15565
W20260926 19:59:46.184827 15565 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.789s user 0.602s sys 0.973s
W20260926 19:59:46.186059 15565 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.791s user 0.602s sys 0.974s
W20260926 19:59:46.186573 15585 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1790 milliseconds
W20260926 19:59:44.396070 15584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:59:46.187093 15586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:59:46.187377 15565 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260926 19:59:46.191830 15565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:59:46.194254 15565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:59:46.195595 15565 hybrid_clock.cc:648] HybridClock initialized: now 1790452786195580 us; error 49 us; skew 500 ppm
I20260926 19:59:46.196444 15565 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:59:46.202571 15565 webserver.cc:533] Webserver started at http://127.11.233.253:39383/ using document root <none> and password file <none>
I20260926 19:59:46.203490 15565 fs_manager.cc:362] Metadata directory not provided
I20260926 19:59:46.203734 15565 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:59:46.214689 15565 fs_manager.cc:714] Time spent opening directory manager: real 0.007s user 0.007s sys 0.001s
I20260926 19:59:46.219519 15614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:59:46.220654 15565 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260926 19:59:46.221021 15565 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data,/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
uuid: "fa6dae08245e46c98e01465e84732c92"
format_stamp: "Formatted at 2026-09-26 19:59:24 on dist-test-slave-jcdj"
I20260926 19:59:46.223007 15565 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:59:46.285413 15565 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260926 19:59:46.287084 15565 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:59:46.287572 15565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:59:46.362913 15565 rpc_server.cc:307] RPC server started. Bound to: 127.11.233.253:39245
I20260926 19:59:46.363018 15666 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.233.253:39245 every 8 connection(s)
I20260926 19:59:46.365865 15565 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/master-1/data/info.pb
I20260926 19:59:46.370707 12199 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskT6OA0Z/build/tsan/bin/kudu as pid 15565
I20260926 19:59:46.379269 15667 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:59:46.385460 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Bootstrap starting.
I20260926 19:59:46.408586 15667 log.cc:826] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Log is configured to *not* fsync() on all Append() calls
I20260926 19:59:46.436101 15671 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader eb0f126418d54df489a4ce95a765b8d3)
I20260926 19:59:46.436575 15671 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:46.438563 15671 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:46.455212 15389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20260926 19:59:46.459923 15389 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:46.485842 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Bootstrap replayed 1/1 log segments. Stats: ops{read=24 overwritten=0 applied=23 ignored=0} inserts{seen=14 ignored=0} mutations{seen=15 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260926 19:59:46.486881 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: Bootstrap complete.
I20260926 19:59:46.498489 15050 heartbeater.cc:344] Connected to a master server at 127.11.233.253:39245
I20260926 19:59:46.513999 15194 heartbeater.cc:344] Connected to a master server at 127.11.233.253:39245
I20260926 19:59:46.517458 15667 raft_consensus.cc:359] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:46.527900 15667 raft_consensus.cc:740] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa6dae08245e46c98e01465e84732c92, State: Initialized, Role: FOLLOWER
I20260926 19:59:46.528931 15667 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 4.24, Last appended by leader: 24, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:46.531636 15679 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:46.532529 15679 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:46.533185 15667 sys_catalog.cc:565] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:59:46.533363 15642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92" is_pre_election: true
I20260926 19:59:46.534330 15642 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c8ef2c095f418083f33f8f0a734c3a in term 4.
I20260926 19:59:46.535619 15386 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92; no voters: eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:46.536293 15671 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260926 19:59:46.536646 15671 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Starting leader election (detected failure of leader eb0f126418d54df489a4ce95a765b8d3)
I20260926 19:59:46.536924 15671 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 5 FOLLOWER]: Advancing to term 6
I20260926 19:59:46.542130 15671 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:46.545193 15642 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c8ef2c095f418083f33f8f0a734c3a" candidate_term: 6 candidate_status { last_received { term: 5 index: 33 } } ignore_live_leader: false dest_uuid: "fa6dae08245e46c98e01465e84732c92"
I20260926 19:59:46.545631 15671 leader_election.cc:290] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 election: Requested vote from peers fa6dae08245e46c98e01465e84732c92 (127.11.233.253:39245), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
I20260926 19:59:46.545850 15642 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 4 FOLLOWER]: Advancing to term 6
W20260926 19:59:46.548434 15389 leader_election.cc:336] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:46.557147 15642 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c8ef2c095f418083f33f8f0a734c3a in term 6.
I20260926 19:59:46.558552 15386 leader_election.cc:304] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c8ef2c095f418083f33f8f0a734c3a, fa6dae08245e46c98e01465e84732c92; no voters: eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:46.560034 15671 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 6 FOLLOWER]: Leader election won for term 6
I20260926 19:59:46.561475 15671 raft_consensus.cc:697] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [term 6 LEADER]: Becoming Leader. State: Replica: 26c8ef2c095f418083f33f8f0a734c3a, State: Running, Role: LEADER
I20260926 19:59:46.562690 15671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 5.33, Last appended by leader: 33, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:46.567983 15692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c8ef2c095f418083f33f8f0a734c3a. Latest consensus state: current_term: 6 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:46.568740 15692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: LEADER
I20260926 19:59:46.589570 15699 catalog_manager.cc:1524] Loading table and tablet metadata into memory...
I20260926 19:59:46.596590 15694 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:46.596870 15694 catalog_manager.cc:1601] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: loading cluster ID for follower catalog manager: success
I20260926 19:59:46.600445 15699 catalog_manager.cc:692] Loaded metadata for table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]
I20260926 19:59:46.602339 15699 catalog_manager.cc:692] Loaded metadata for table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]
I20260926 19:59:46.604576 15699 catalog_manager.cc:692] Loaded metadata for table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]
I20260926 19:59:46.604772 15694 catalog_manager.cc:1623] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: acquiring CA information for follower catalog manager: success
I20260926 19:59:46.611049 15694 catalog_manager.cc:1651] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:59:46.619823 15699 tablet_loader.cc:96] loaded metadata for tablet 0c8337dd2c7440c08ca282be7496806e (table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792])
I20260926 19:59:46.621212 15699 tablet_loader.cc:96] loaded metadata for tablet 197039cbece04593bc7b11041dbf1c8e (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb])
I20260926 19:59:46.623613 15699 tablet_loader.cc:96] loaded metadata for tablet 7171f772b30648afba52e793d0d0228b (table default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875])
I20260926 19:59:46.625021 15699 catalog_manager.cc:1533] Initializing Kudu cluster ID...
I20260926 19:59:46.634912 15699 catalog_manager.cc:1308] Loaded cluster ID: b4293131cc434d2194a1cdf2d39da0b5
I20260926 19:59:46.635214 15699 catalog_manager.cc:1544] Initializing Kudu internal certificate authority...
I20260926 19:59:46.651100 15699 catalog_manager.cc:1553] Loading token signing keys...
I20260926 19:59:46.655803 15699 catalog_manager.cc:6230] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Loaded TSK: 0
I20260926 19:59:46.657047 15699 catalog_manager.cc:1563] Initializing in-progress tserver states...
I20260926 19:59:46.662755 15699 catalog_manager.cc:1576] Loading latest processed Hive Metastore notification log event ID...
I20260926 19:59:46.667161 15699 catalog_manager.cc:6068] Last processed Hive Metastore notification event ID: -1
I20260926 19:59:46.644210 15565 master_runner.cc:187] Error getting master registration for 127.11.233.252:37917: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111),
I20260926 19:59:46.672255 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:59:46.686554 15642 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Refusing update from remote peer 26c8ef2c095f418083f33f8f0a734c3a: Log matching property violated. Preceding OpId in replica: term: 4 index: 24. Preceding OpId from leader: term: 6 index: 35. (index mismatch)
I20260926 19:59:46.688243 15671 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 23, Time since last communication: 0.000s
W20260926 19:59:46.688962 15389 consensus_peers.cc:597] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a -> Peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Couldn't send request to peer eb0f126418d54df489a4ce95a765b8d3. Status: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 19:59:46.729274 15400 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39838:
name: "default.Table_2369b73124a04219b46431eeadde2ac7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:46.729796 15399 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39838:
name: "default.Table_048abbb957574bd9904e0ba925421cee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:46.730553 15401 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39838:
name: "default.Table_3c580a21625b41a4894dd8072db5ba09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:46.731302 15398 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39838:
table { table_name: "default.Table_7a1c4598445a4ac5845670a68237cb86" } new_table_name: "default.Table_2d6e52c75c4f4413a0185f462c80af99" modify_external_catalogs: true
I20260926 19:59:46.732512 15397 master_service.cc:960] ReplaceTablet: received request to replace tablet 0c8337dd2c7440c08ca282be7496806e from {username='slave'} at 127.0.0.1:39838
I20260926 19:59:46.739292 15679 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c8ef2c095f418083f33f8f0a734c3a. Latest consensus state: current_term: 6 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:46.740051 15679 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:46.748540 14905 heartbeater.cc:344] Connected to a master server at 127.11.233.253:39245
I20260926 19:59:46.749423 15395 master_service.cc:960] ReplaceTablet: received request to replace tablet 197039cbece04593bc7b11041dbf1c8e from {username='slave'} at 127.0.0.1:39838
I20260926 19:59:46.751873 15678 mvcc.cc:204] Tried to move back new op lower bound from 7333694595857223680 to 7333694591448285184. Current Snapshot: MvccSnapshot[applied={T|T < 7333694595857223680 or (T in {7333694595857223680})}]
F20260926 19:59:46.755344 15455 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 4 received events: NotificationEvent(eventId=1, eventTime=1790452776, eventType=CREATE_TABLE, dbName=default, tableName=table_131b6490764e47dc8a7a589300bbf272, message={"server":"","servicePrincipal":"","db":"default","table":"table_131b6490764e47dc8a7a589300bbf272","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_131b6490764e47dc8a7a589300bbf272\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790452775},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/warehouse/table_131b6490764e47dc8a7a589300bbf272\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_131b6490764e47dc8a7a589300bbf272\",\"kudu.table_id\":\"53eb5bec635046daa62c8b441e0d404c\",\"transient_lastDdlTime\":\"1790452775\",\"comment\":\"\",\"kudu.master_addresses\":\"127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"b4293131cc434d2194a1cdf2d39da0b5\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790452776,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1790452776, eventType=CREATE_TABLE, dbName=default, tableName=table_214fa130d427469eaf37415c0767c7e3, message={"server":"","servicePrincipal":"","db":"default","table":"table_214fa130d427469eaf37415c0767c7e3","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_214fa130d427469eaf37415c0767c7e3\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1790452776},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/warehouse/table_214fa130d427469eaf37415c0767c7e3\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_214fa130d427469eaf37415c0767c7e3\",\"kudu.table_id\":\"a07796662b5d4b0b8b019aa4e111b875\",\"transient_lastDdlTime\":\"1790452776\",\"comment\":\"\",\"kudu.master_addresses\":\"127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"b4293131cc434d2194a1cdf2d39da0b5\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1790452776,"files":[]}, messageFormat=json-0.2, catName=hive);
I20260926 19:59:46.758391 15671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:46.760422 15671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a [sys.catalog]: This master's current role is: LEADER
*** Check failure stack trace: ***
I20260926 19:59:46.764326 15679 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:46.765033 15679 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:46.765913 15706 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "26c8ef2c095f418083f33f8f0a734c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } } }
I20260926 19:59:46.766654 15706 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:59:46.769527 15632 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" instance_seqno: 1790452770962163) as {username='slave'} at 127.11.233.193:46061; Asking this server to re-register.
I20260926 19:59:46.771617 15394 catalog_manager.cc:2546] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39838:
table { table_name: "default.Table_e144b6535e8448a3b04ecd97e88b06a9" } modify_external_catalogs: true
I20260926 19:59:46.772531 15394 catalog_manager.cc:2794] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39838:
table { table_name: "default.Table_e144b6535e8448a3b04ecd97e88b06a9" } modify_external_catalogs: true
I20260926 19:59:46.774793 14905 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:46.759490 15396 catalog_manager.cc:2296] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39838:
name: "default.Table_c74a17cf08a9482ab2915e2d7ff9fbeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:59:46.779965 15402 catalog_manager.cc:3583] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39838:
table { table_name: "default.Table_214fa130d427469eaf37415c0767c7e3" } new_table_name: "default.Table_110ad590096942f0a032e632ccedc05f" modify_external_catalogs: true
I20260926 19:59:46.779120 15632 ts_manager.cc:194] Registered new tserver with Master: 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755)
@ 0x7fa162b3a920 google::LogMessage::Flush() at ??:0
I20260926 19:59:46.790722 15397 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Sending DeleteTablet for 3 replicas of tablet 0c8337dd2c7440c08ca282be7496806e
I20260926 19:59:46.809428 15397 catalog_manager.cc:6934] ReplaceTablet: tablet 0c8337dd2c7440c08ca282be7496806e deleted and replaced by tablet 1dab21ccfef842e49c1242695a59f5ea
I20260926 19:59:46.803970 15395 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Sending DeleteTablet for 3 replicas of tablet 197039cbece04593bc7b11041dbf1c8e
I20260926 19:59:46.815218 15395 catalog_manager.cc:6934] ReplaceTablet: tablet 197039cbece04593bc7b11041dbf1c8e deleted and replaced by tablet 535d135b220c4855b687e7315aaaa399
I20260926 19:59:46.828514 14840 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c8337dd2c7440c08ca282be7496806e with delete_type TABLET_DATA_DELETED (replaced by tablet 1dab21ccfef842e49c1242695a59f5ea) from {username='slave'} at 127.0.0.1:49434
I20260926 19:59:46.829450 14839 tablet_service.cc:1558] Processing DeleteTablet for tablet 197039cbece04593bc7b11041dbf1c8e with delete_type TABLET_DATA_DELETED (replaced by tablet 535d135b220c4855b687e7315aaaa399) from {username='slave'} at 127.0.0.1:49434
@ 0x7fa162b3a68f google::LogMessage::~LogMessage() at ??:0
I20260926 19:59:46.838364 14983 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c8337dd2c7440c08ca282be7496806e with delete_type TABLET_DATA_DELETED (replaced by tablet 1dab21ccfef842e49c1242695a59f5ea) from {username='slave'} at 127.0.0.1:50906
I20260926 19:59:46.839182 14982 tablet_service.cc:1558] Processing DeleteTablet for tablet 197039cbece04593bc7b11041dbf1c8e with delete_type TABLET_DATA_DELETED (replaced by tablet 535d135b220c4855b687e7315aaaa399) from {username='slave'} at 127.0.0.1:50906
I20260926 19:59:46.849591 14839 tablet_service.cc:1511] Processing CreateTablet for tablet 1dab21ccfef842e49c1242695a59f5ea (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:46.850747 14839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dab21ccfef842e49c1242695a59f5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:46.854755 14982 tablet_service.cc:1511] Processing CreateTablet for tablet 1dab21ccfef842e49c1242695a59f5ea (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:46.855875 14982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dab21ccfef842e49c1242695a59f5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:46.856442 15719 tablet_replica.cc:333] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:46.857322 15719 raft_consensus.cc:2243] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:46.857909 15719 raft_consensus.cc:2272] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:46.860285 15128 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c8337dd2c7440c08ca282be7496806e with delete_type TABLET_DATA_DELETED (replaced by tablet 1dab21ccfef842e49c1242695a59f5ea) from {username='slave'} at 127.0.0.1:48696
I20260926 19:59:46.860565 15129 tablet_service.cc:1558] Processing DeleteTablet for tablet 197039cbece04593bc7b11041dbf1c8e with delete_type TABLET_DATA_DELETED (replaced by tablet 535d135b220c4855b687e7315aaaa399) from {username='slave'} at 127.0.0.1:48696
I20260926 19:59:46.862560 15129 tablet_service.cc:1511] Processing CreateTablet for tablet 1dab21ccfef842e49c1242695a59f5ea (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:46.864436 15129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dab21ccfef842e49c1242695a59f5ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:46.866343 15719 ts_tablet_manager.cc:1916] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:46.875200 15717 tablet_replica.cc:333] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:46.875195 15127 tablet_service.cc:1511] Processing CreateTablet for tablet 535d135b220c4855b687e7315aaaa399 (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:46.875993 15717 raft_consensus.cc:2243] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:59:46.876330 15127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535d135b220c4855b687e7315aaaa399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:46.877024 15717 raft_consensus.cc:2272] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:46.880491 14840 tablet_service.cc:1511] Processing CreateTablet for tablet 535d135b220c4855b687e7315aaaa399 (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:46.881587 14840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535d135b220c4855b687e7315aaaa399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:46.878364 15720 tablet_replica.cc:333] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:46.886314 15720 raft_consensus.cc:2243] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:46.887081 15720 raft_consensus.cc:2272] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7fa16dccd35c kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260926 19:59:46.889468 15720 ts_tablet_manager.cc:1916] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:46.892410 14983 tablet_service.cc:1511] Processing CreateTablet for tablet 535d135b220c4855b687e7315aaaa399 (DEFAULT_TABLE table=default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:46.893553 14983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 535d135b220c4855b687e7315aaaa399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:46.894697 15722 tablet_bootstrap.cc:492] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:46.899113 15717 ts_tablet_manager.cc:1916] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:46.901336 15722 tablet_bootstrap.cc:654] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:46.915707 15723 tablet_bootstrap.cc:492] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:46.917205 15721 tablet_bootstrap.cc:492] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:46.917766 15719 ts_tablet_manager.cc:1929] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:46.918174 15719 log.cc:1199] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/0c8337dd2c7440c08ca282be7496806e
I20260926 19:59:46.919030 15719 ts_tablet_manager.cc:1950] T 0c8337dd2c7440c08ca282be7496806e P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
@ 0x7fa16dcccb1c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20260926 19:59:46.929049 15723 tablet_bootstrap.cc:654] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:46.930492 15722 tablet_bootstrap.cc:492] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:46.930903 15722 ts_tablet_manager.cc:1403] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.004s
I20260926 19:59:46.933761 15722 raft_consensus.cc:359] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:46.934566 15722 raft_consensus.cc:385] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:46.934931 15722 raft_consensus.cc:740] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:46.935746 15722 consensus_queue.cc:260] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:46.938441 15387 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet 0c8337dd2c7440c08ca282be7496806e (table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]) successfully deleted
I20260926 19:59:46.939484 15719 tablet_replica.cc:333] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:46.940254 15719 raft_consensus.cc:2243] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:59:46.941210 15719 raft_consensus.cc:2272] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:46.943287 15719 ts_tablet_manager.cc:1916] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:46.944470 15632 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" instance_seqno: 1790452775589282) as {username='slave'} at 127.11.233.195:51733; Asking this server to re-register.
I20260926 19:59:46.947548 15720 ts_tablet_manager.cc:1929] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:46.952284 15721 tablet_bootstrap.cc:654] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:46.950742 15194 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:46.954041 15720 log.cc:1199] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/0c8337dd2c7440c08ca282be7496806e
I20260926 19:59:46.955075 15720 ts_tablet_manager.cc:1950] T 0c8337dd2c7440c08ca282be7496806e P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:46.951694 15722 ts_tablet_manager.cc:1434] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260926 19:59:46.956869 15196 heartbeater.cc:499] Master 127.11.233.254:32801 was elected leader, sending a full tablet report...
I20260926 19:59:46.958009 15720 tablet_replica.cc:333] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:46.958004 15632 ts_manager.cc:194] Registered new tserver with Master: e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633)
I20260926 19:59:46.958760 15720 raft_consensus.cc:2243] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:46.959290 15720 raft_consensus.cc:2272] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:46.960556 15389 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet 0c8337dd2c7440c08ca282be7496806e (table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]) successfully deleted
W20260926 19:59:46.962430 15395 catalog_manager.cc:5671] Ignoring report from unknown tablet 21847f1f910b458fad550d9ef426173e
I20260926 19:59:46.963914 15722 tablet_bootstrap.cc:492] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:46.967882 15395 catalog_manager.cc:5838] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193). New cstate: current_term: 1 leader_uuid: "56196546dad349b9bd344c4e0b6a6874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } }
I20260926 19:59:46.969841 15723 tablet_bootstrap.cc:492] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:46.970341 15723 ts_tablet_manager.cc:1403] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.055s user 0.016s sys 0.004s
I20260926 19:59:46.972687 15722 tablet_bootstrap.cc:654] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:46.973156 15723 raft_consensus.cc:359] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:46.974044 15723 raft_consensus.cc:385] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:46.974376 15723 raft_consensus.cc:740] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:46.975271 15723 consensus_queue.cc:260] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:46.980706 15717 ts_tablet_manager.cc:1929] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:46.983757 15717 log.cc:1199] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/0c8337dd2c7440c08ca282be7496806e
I20260926 19:59:46.984728 15717 ts_tablet_manager.cc:1950] T 0c8337dd2c7440c08ca282be7496806e P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:46.979316 15720 ts_tablet_manager.cc:1916] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:46.990993 15721 tablet_bootstrap.cc:492] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:46.992148 15632 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" instance_seqno: 1790452773079792) as {username='slave'} at 127.11.233.194:55259; Asking this server to re-register.
I20260926 19:59:46.992643 15721 ts_tablet_manager.cc:1403] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.119s user 0.010s sys 0.003s
I20260926 19:59:46.993700 15050 heartbeater.cc:461] Registering TS with master...
I20260926 19:59:46.994954 15722 tablet_bootstrap.cc:492] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:46.995468 15722 ts_tablet_manager.cc:1403] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.011s
I20260926 19:59:46.995726 15721 raft_consensus.cc:359] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:46.997961 15632 ts_manager.cc:194] Registered new tserver with Master: 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:46.998276 15721 raft_consensus.cc:385] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:46.998567 15721 raft_consensus.cc:740] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:46.998389 15722 raft_consensus.cc:359] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
@ 0x7fa16dcd0b92 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260926 19:59:47.000528 15722 raft_consensus.cc:385] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:47.000823 15722 raft_consensus.cc:740] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.001660 15719 ts_tablet_manager.cc:1929] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:47.001870 15723 ts_tablet_manager.cc:1434] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260926 19:59:47.005638 15052 heartbeater.cc:499] Master 127.11.233.254:32801 was elected leader, sending a full tablet report...
I20260926 19:59:47.002090 15719 log.cc:1199] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/197039cbece04593bc7b11041dbf1c8e
I20260926 19:59:47.006981 15719 ts_tablet_manager.cc:1950] T 197039cbece04593bc7b11041dbf1c8e P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
W20260926 19:59:47.011180 15397 catalog_manager.cc:5671] Ignoring report from unknown tablet 21847f1f910b458fad550d9ef426173e
I20260926 19:59:47.012562 15723 tablet_bootstrap.cc:492] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:47.014786 15389 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet 0c8337dd2c7440c08ca282be7496806e (table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]) successfully deleted
I20260926 19:59:47.015636 15721 consensus_queue.cc:260] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.021836 15722 consensus_queue.cc:260] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.023454 15723 tablet_bootstrap.cc:654] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:47.024887 15717 tablet_replica.cc:333] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:47.025687 15717 raft_consensus.cc:2243] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:47.026192 15717 raft_consensus.cc:2272] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:47.026361 15393 master_service.cc:960] ReplaceTablet: received request to replace tablet 1dab21ccfef842e49c1242695a59f5ea from {username='slave'} at 127.0.0.1:39838
I20260926 19:59:47.028157 15717 ts_tablet_manager.cc:1916] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:47.031577 15722 ts_tablet_manager.cc:1434] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.036s user 0.000s sys 0.005s
I20260926 19:59:47.047477 15387 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet 197039cbece04593bc7b11041dbf1c8e (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:47.053591 15721 ts_tablet_manager.cc:1434] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.060s user 0.006s sys 0.007s
I20260926 19:59:47.056190 15720 ts_tablet_manager.cc:1929] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:47.056627 15720 log.cc:1199] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/197039cbece04593bc7b11041dbf1c8e
I20260926 19:59:47.057370 15720 ts_tablet_manager.cc:1950] T 197039cbece04593bc7b11041dbf1c8e P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:47.060657 15389 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet 197039cbece04593bc7b11041dbf1c8e (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:47.064530 15723 tablet_bootstrap.cc:492] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:47.061626 14909 heartbeater.cc:499] Master 127.11.233.254:32801 was elected leader, sending a full tablet report...
I20260926 19:59:47.072410 15721 tablet_bootstrap.cc:492] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:47.067795 15395 master_service.cc:960] ReplaceTablet: received request to replace tablet 7171f772b30648afba52e793d0d0228b from {username='slave'} at 127.0.0.1:39838
I20260926 19:59:47.075139 15723 ts_tablet_manager.cc:1403] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.008s
@ 0x7fa16dcd0b4a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260926 19:59:47.080133 15723 raft_consensus.cc:359] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.080941 15723 raft_consensus.cc:385] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:47.084268 15717 ts_tablet_manager.cc:1929] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:47.084689 15717 log.cc:1199] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/197039cbece04593bc7b11041dbf1c8e
I20260926 19:59:47.085469 15717 ts_tablet_manager.cc:1950] T 197039cbece04593bc7b11041dbf1c8e P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:47.086251 15721 tablet_bootstrap.cc:654] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:47.081300 15723 raft_consensus.cc:740] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.088169 15389 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet 197039cbece04593bc7b11041dbf1c8e (table default.table_7a1c4598445a4ac5845670a68237cb86 [id=35c3d430e5a545678edb506779d24bdb]) successfully deleted
I20260926 19:59:47.088323 15723 consensus_queue.cc:260] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.099828 15723 ts_tablet_manager.cc:1434] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.024s user 0.005s sys 0.003s
I20260926 19:59:47.109057 15393 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Sending DeleteTablet for 3 replicas of tablet 1dab21ccfef842e49c1242695a59f5ea
@ 0x7fa16dcd0ada std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260926 19:59:47.113574 14840 tablet_service.cc:1558] Processing DeleteTablet for tablet 1dab21ccfef842e49c1242695a59f5ea with delete_type TABLET_DATA_DELETED (replaced by tablet 94b81663f0b14be39a3be2d13035f3f4) from {username='slave'} at 127.0.0.1:49434
I20260926 19:59:47.114378 15717 tablet_replica.cc:333] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:47.114380 15127 tablet_service.cc:1558] Processing DeleteTablet for tablet 1dab21ccfef842e49c1242695a59f5ea with delete_type TABLET_DATA_DELETED (replaced by tablet 94b81663f0b14be39a3be2d13035f3f4) from {username='slave'} at 127.0.0.1:48696
I20260926 19:59:47.115204 15717 raft_consensus.cc:2243] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:47.115715 15717 raft_consensus.cc:2272] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:47.116550 15720 tablet_replica.cc:333] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:47.117385 15720 raft_consensus.cc:2243] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:47.117874 15720 raft_consensus.cc:2272] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:47.119915 15720 ts_tablet_manager.cc:1916] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:47.121316 14982 tablet_service.cc:1558] Processing DeleteTablet for tablet 1dab21ccfef842e49c1242695a59f5ea with delete_type TABLET_DATA_DELETED (replaced by tablet 94b81663f0b14be39a3be2d13035f3f4) from {username='slave'} at 127.0.0.1:50906
I20260926 19:59:47.122038 15719 tablet_replica.cc:333] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:47.122756 15719 raft_consensus.cc:2243] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:47.123205 15719 raft_consensus.cc:2272] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:47.123793 15393 catalog_manager.cc:6934] ReplaceTablet: tablet 1dab21ccfef842e49c1242695a59f5ea deleted and replaced by tablet 94b81663f0b14be39a3be2d13035f3f4
I20260926 19:59:47.125211 15717 ts_tablet_manager.cc:1916] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:47.127466 15719 ts_tablet_manager.cc:1916] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:47.132190 15721 tablet_bootstrap.cc:492] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:47.132603 15721 ts_tablet_manager.cc:1403] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.060s user 0.012s sys 0.000s
I20260926 19:59:47.138711 15721 raft_consensus.cc:359] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.139387 15721 raft_consensus.cc:385] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260926 19:59:47.144095 15397 catalog_manager.cc:5671] Ignoring report from unknown tablet 197039cbece04593bc7b11041dbf1c8e
W20260926 19:59:47.144616 15397 catalog_manager.cc:5671] Ignoring report from unknown tablet 21847f1f910b458fad550d9ef426173e
I20260926 19:59:47.154181 15395 catalog_manager.cc:6133] T 00000000000000000000000000000000 P 26c8ef2c095f418083f33f8f0a734c3a: Sending DeleteTablet for 3 replicas of tablet 7171f772b30648afba52e793d0d0228b
@ 0x7fa16dcd0aa2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260926 19:59:47.155761 15395 catalog_manager.cc:6934] ReplaceTablet: tablet 7171f772b30648afba52e793d0d0228b deleted and replaced by tablet c86b5abebbe949049fccd1a684b5c279
I20260926 19:59:47.156087 14840 tablet_service.cc:1558] Processing DeleteTablet for tablet 7171f772b30648afba52e793d0d0228b with delete_type TABLET_DATA_DELETED (replaced by tablet c86b5abebbe949049fccd1a684b5c279) from {username='slave'} at 127.0.0.1:49434
I20260926 19:59:47.158384 14982 tablet_service.cc:1558] Processing DeleteTablet for tablet 7171f772b30648afba52e793d0d0228b with delete_type TABLET_DATA_DELETED (replaced by tablet c86b5abebbe949049fccd1a684b5c279) from {username='slave'} at 127.0.0.1:50906
I20260926 19:59:47.161132 15127 tablet_service.cc:1558] Processing DeleteTablet for tablet 7171f772b30648afba52e793d0d0228b with delete_type TABLET_DATA_DELETED (replaced by tablet c86b5abebbe949049fccd1a684b5c279) from {username='slave'} at 127.0.0.1:48696
I20260926 19:59:47.163878 15720 ts_tablet_manager.cc:1929] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:47.164237 15720 log.cc:1199] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/1dab21ccfef842e49c1242695a59f5ea
I20260926 19:59:47.164882 15720 ts_tablet_manager.cc:1950] T 1dab21ccfef842e49c1242695a59f5ea P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:47.173185 15717 ts_tablet_manager.cc:1929] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:47.139644 15721 raft_consensus.cc:740] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.173658 15717 log.cc:1199] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/1dab21ccfef842e49c1242695a59f5ea
I20260926 19:59:47.174429 15721 consensus_queue.cc:260] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.176743 15389 catalog_manager.cc:5063] TS e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633): tablet 1dab21ccfef842e49c1242695a59f5ea (table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]) successfully deleted
I20260926 19:59:47.179862 15721 ts_tablet_manager.cc:1434] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.047s user 0.004s sys 0.002s
I20260926 19:59:47.174424 15717 ts_tablet_manager.cc:1950] T 1dab21ccfef842e49c1242695a59f5ea P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:47.188540 15720 tablet_replica.cc:333] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: stopping tablet replica
I20260926 19:59:47.189288 15720 raft_consensus.cc:2243] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:47.189845 15720 raft_consensus.cc:2272] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:47.191823 15720 ts_tablet_manager.cc:1916] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:47.193109 15389 catalog_manager.cc:5063] TS 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): tablet 1dab21ccfef842e49c1242695a59f5ea (table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]) successfully deleted
I20260926 19:59:47.193966 14982 tablet_service.cc:1511] Processing CreateTablet for tablet 94b81663f0b14be39a3be2d13035f3f4 (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:47.195080 14982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94b81663f0b14be39a3be2d13035f3f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:47.194890 14840 tablet_service.cc:1511] Processing CreateTablet for tablet 94b81663f0b14be39a3be2d13035f3f4 (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:47.195997 14840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94b81663f0b14be39a3be2d13035f3f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:47.200595 15127 tablet_service.cc:1511] Processing CreateTablet for tablet 94b81663f0b14be39a3be2d13035f3f4 (DEFAULT_TABLE table=default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:47.201865 15127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94b81663f0b14be39a3be2d13035f3f4. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7fa16dccfd9e std::__1::__function::__func<>::operator()() at ??:0
@ 0x5bd4e5 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5bd31a std::__1::function<>::operator()() at ??:?
I20260926 19:59:47.216089 15719 ts_tablet_manager.cc:1929] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260926 19:59:47.216499 15719 log.cc:1199] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/1dab21ccfef842e49c1242695a59f5ea
I20260926 19:59:47.217139 15719 ts_tablet_manager.cc:1950] T 1dab21ccfef842e49c1242695a59f5ea P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
I20260926 19:59:47.221276 15717 tablet_replica.cc:333] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: stopping tablet replica
I20260926 19:59:47.222199 15717 raft_consensus.cc:2243] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:59:47.222991 15387 catalog_manager.cc:5063] TS 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): tablet 1dab21ccfef842e49c1242695a59f5ea (table default.table_e144b6535e8448a3b04ecd97e88b06a9 [id=ed2f37ba144d40c5a4751dc80b67c792]) successfully deleted
I20260926 19:59:47.223201 15717 raft_consensus.cc:2272] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7fa164d3081b kudu::Thread::SuperviseThread() at ??:0
@ 0x4569a0 __tsan_thread_start_func at ??:?
I20260926 19:59:47.225342 15717 ts_tablet_manager.cc:1916] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:59:47.240569 14983 tablet_service.cc:1511] Processing CreateTablet for tablet c86b5abebbe949049fccd1a684b5c279 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:47.241720 14983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86b5abebbe949049fccd1a684b5c279. 1 dirs total, 0 dirs full, 0 dirs failed
W20260926 19:59:47.250408 15389 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.233.252:37917, user_credentials={real_user=slave}} blocked reactor thread for 45176.7us
I20260926 19:59:47.250679 15721 tablet_bootstrap.cc:492] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
@ 0x7fa1664916db start_thread at ??:0
I20260926 19:59:47.258105 15723 tablet_bootstrap.cc:492] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:47.258587 15721 tablet_bootstrap.cc:654] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:47.259578 15129 tablet_service.cc:1511] Processing CreateTablet for tablet c86b5abebbe949049fccd1a684b5c279 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:47.260730 15129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86b5abebbe949049fccd1a684b5c279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:47.264570 15723 tablet_bootstrap.cc:654] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:47.270129 14840 tablet_service.cc:1511] Processing CreateTablet for tablet c86b5abebbe949049fccd1a684b5c279 (DEFAULT_TABLE table=default.table_214fa130d427469eaf37415c0767c7e3 [id=a07796662b5d4b0b8b019aa4e111b875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:59:47.271273 14840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c86b5abebbe949049fccd1a684b5c279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:59:47.273641 15719 tablet_replica.cc:333] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: stopping tablet replica
I20260926 19:59:47.274459 15719 raft_consensus.cc:2243] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:59:47.274987 15719 raft_consensus.cc:2272] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:59:47.276983 15719 ts_tablet_manager.cc:1916] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fa1617dc61f clone at ??:0
I20260926 19:59:47.286799 15722 tablet_bootstrap.cc:492] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:47.293001 15722 tablet_bootstrap.cc:654] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:47.303607 15722 tablet_bootstrap.cc:492] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:47.304023 15722 ts_tablet_manager.cc:1403] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.000s
I20260926 19:59:47.306658 15722 raft_consensus.cc:359] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.307303 15722 raft_consensus.cc:385] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:47.307561 15722 raft_consensus.cc:740] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.311313 15722 consensus_queue.cc:260] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.314955 15720 ts_tablet_manager.cc:1929] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:47.315375 15720 log.cc:1199] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-2/wal/wals/7171f772b30648afba52e793d0d0228b
I20260926 19:59:47.315505 15723 tablet_bootstrap.cc:492] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:47.315936 15723 ts_tablet_manager.cc:1403] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.013s
I20260926 19:59:47.316169 15720 ts_tablet_manager.cc:1950] T 7171f772b30648afba52e793d0d0228b P e3e2a17ea4c94765be709a7ec8af4c8a: Deleting consensus metadata
I20260926 19:59:47.318637 15723 raft_consensus.cc:359] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.319309 15723 raft_consensus.cc:385] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:47.319578 15723 raft_consensus.cc:740] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.320341 15717 ts_tablet_manager.cc:1929] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:47.320755 15717 log.cc:1199] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-0/wal/wals/7171f772b30648afba52e793d0d0228b
I20260926 19:59:47.322537 15723 consensus_queue.cc:260] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.327205 15717 ts_tablet_manager.cc:1950] T 7171f772b30648afba52e793d0d0228b P 56196546dad349b9bd344c4e0b6a6874: Deleting consensus metadata
I20260926 19:59:47.327675 15721 tablet_bootstrap.cc:492] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:47.328091 15721 ts_tablet_manager.cc:1403] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.078s user 0.008s sys 0.017s
I20260926 19:59:47.329893 15723 ts_tablet_manager.cc:1434] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.014s user 0.006s sys 0.002s
I20260926 19:59:47.330780 15721 raft_consensus.cc:359] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.331410 15721 raft_consensus.cc:385] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:47.334910 15722 ts_tablet_manager.cc:1434] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.031s user 0.008s sys 0.000s
I20260926 19:59:47.338248 15723 tablet_bootstrap.cc:492] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9: Bootstrap starting.
I20260926 19:59:47.343899 15722 tablet_bootstrap.cc:492] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a: Bootstrap starting.
I20260926 19:59:47.344368 15723 tablet_bootstrap.cc:654] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:47.346684 15700 raft_consensus.cc:493] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:47.347138 15700 raft_consensus.cc:515] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.349140 15700 leader_election.cc:290] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755), 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:47.349375 15719 ts_tablet_manager.cc:1929] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260926 19:59:47.349756 15719 log.cc:1199] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting WAL directory at /tmp/dist-test-taskT6OA0Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790452699499916-12199-0/minicluster-data/ts-1/wal/wals/7171f772b30648afba52e793d0d0228b
I20260926 19:59:47.350464 15719 ts_tablet_manager.cc:1950] T 7171f772b30648afba52e793d0d0228b P 17fec3df2f014ed5b1803b8a2bc38df9: Deleting consensus metadata
I20260926 19:59:47.360471 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5abebbe949049fccd1a684b5c279" candidate_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874" is_pre_election: true
I20260926 19:59:47.361878 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5abebbe949049fccd1a684b5c279" candidate_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" is_pre_election: true
W20260926 19:59:47.364305 15085 leader_election.cc:343] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755): Illegal state: must be running to vote when last-logged opid is not known
W20260926 19:59:47.366215 15083 leader_election.cc:343] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597): Illegal state: must be running to vote when last-logged opid is not known
I20260926 19:59:47.366614 15083 leader_election.cc:304] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3e2a17ea4c94765be709a7ec8af4c8a; no voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874
I20260926 19:59:47.369035 15700 raft_consensus.cc:2749] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:59:47.331804 15721 raft_consensus.cc:740] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.384111 15723 tablet_bootstrap.cc:492] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9: No bootstrap required, opened a new log
I20260926 19:59:47.384567 15723 ts_tablet_manager.cc:1403] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.006s
I20260926 19:59:47.384339 15721 consensus_queue.cc:260] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.387207 15723 raft_consensus.cc:359] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.387848 15723 raft_consensus.cc:385] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:47.388115 15723 raft_consensus.cc:740] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.388804 15723 consensus_queue.cc:260] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.395057 15721 ts_tablet_manager.cc:1434] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.067s user 0.005s sys 0.001s
I20260926 19:59:47.401950 15732 raft_consensus.cc:493] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:47.402417 15732 raft_consensus.cc:515] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.404363 15732 leader_election.cc:290] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633), 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:47.406678 15723 ts_tablet_manager.cc:1434] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
I20260926 19:59:47.406780 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535d135b220c4855b687e7315aaaa399" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" is_pre_election: true
I20260926 19:59:47.407387 15004 raft_consensus.cc:2468] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:47.408485 14793 leader_election.cc:304] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:47.410148 15732 raft_consensus.cc:2804] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:47.410470 15732 raft_consensus.cc:493] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:47.410792 15732 raft_consensus.cc:3060] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.410835 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535d135b220c4855b687e7315aaaa399" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" is_pre_election: true
I20260926 19:59:47.411592 15149 raft_consensus.cc:2468] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:47.433331 15721 tablet_bootstrap.cc:492] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874: Bootstrap starting.
I20260926 19:59:47.442282 15722 tablet_bootstrap.cc:654] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:47.443416 15732 raft_consensus.cc:515] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.447448 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535d135b220c4855b687e7315aaaa399" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
I20260926 19:59:47.448064 15149 raft_consensus.cc:3060] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.450934 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "535d135b220c4855b687e7315aaaa399" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
I20260926 19:59:47.451511 15004 raft_consensus.cc:3060] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.453151 15721 tablet_bootstrap.cc:654] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874: Neither blocks nor log segments found. Creating new log.
I20260926 19:59:47.455446 15149 raft_consensus.cc:2468] T 535d135b220c4855b687e7315aaaa399 P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 1.
I20260926 19:59:47.455945 15732 leader_election.cc:290] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Requested vote from peers e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633), 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:47.456580 14795 leader_election.cc:304] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56196546dad349b9bd344c4e0b6a6874, e3e2a17ea4c94765be709a7ec8af4c8a; no voters:
I20260926 19:59:47.458513 15004 raft_consensus.cc:2468] T 535d135b220c4855b687e7315aaaa399 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 1.
I20260926 19:59:47.464576 15732 raft_consensus.cc:2804] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:47.466691 15732 raft_consensus.cc:697] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [term 1 LEADER]: Becoming Leader. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Running, Role: LEADER
I20260926 19:59:47.468683 15732 consensus_queue.cc:237] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.486423 15722 tablet_bootstrap.cc:492] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a: No bootstrap required, opened a new log
I20260926 19:59:47.486969 15722 ts_tablet_manager.cc:1403] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent bootstrapping tablet: real 0.143s user 0.014s sys 0.000s
I20260926 19:59:47.491001 15721 tablet_bootstrap.cc:492] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874: No bootstrap required, opened a new log
I20260926 19:59:47.498065 15721 ts_tablet_manager.cc:1403] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874: Time spent bootstrapping tablet: real 0.066s user 0.012s sys 0.000s
I20260926 19:59:47.495143 15722 raft_consensus.cc:359] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.502223 15722 raft_consensus.cc:385] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:47.508622 15721 raft_consensus.cc:359] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.509403 15722 raft_consensus.cc:740] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3e2a17ea4c94765be709a7ec8af4c8a, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.513741 15722 consensus_queue.cc:260] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.526418 15721 raft_consensus.cc:385] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:59:47.526793 15721 raft_consensus.cc:740] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Initialized, Role: FOLLOWER
I20260926 19:59:47.527524 15721 consensus_queue.cc:260] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.516651 15722 ts_tablet_manager.cc:1434] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260926 19:59:47.542829 15721 ts_tablet_manager.cc:1434] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874: Time spent starting tablet: real 0.044s user 0.005s sys 0.000s
I20260926 19:59:47.576017 15732 raft_consensus.cc:493] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:47.576521 15732 raft_consensus.cc:515] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.578594 15732 leader_election.cc:290] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633), 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:47.579658 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5abebbe949049fccd1a684b5c279" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" is_pre_election: true
I20260926 19:59:47.580197 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5abebbe949049fccd1a684b5c279" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" is_pre_election: true
I20260926 19:59:47.580391 15149 raft_consensus.cc:2468] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:47.580762 15004 raft_consensus.cc:2468] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 0.
I20260926 19:59:47.582003 14793 leader_election.cc:304] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, 56196546dad349b9bd344c4e0b6a6874; no voters:
I20260926 19:59:47.582784 15732 raft_consensus.cc:2804] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:47.583169 15732 raft_consensus.cc:493] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:47.583510 15732 raft_consensus.cc:3060] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.590430 15732 raft_consensus.cc:515] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.592502 15732 leader_election.cc:290] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Requested vote from peers e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633), 17fec3df2f014ed5b1803b8a2bc38df9 (127.11.233.194:34597)
I20260926 19:59:47.593750 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5abebbe949049fccd1a684b5c279" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
I20260926 19:59:47.594439 15149 raft_consensus.cc:3060] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.594399 15004 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c86b5abebbe949049fccd1a684b5c279" candidate_uuid: "56196546dad349b9bd344c4e0b6a6874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17fec3df2f014ed5b1803b8a2bc38df9"
I20260926 19:59:47.594926 15004 raft_consensus.cc:3060] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.602296 15149 raft_consensus.cc:2468] T c86b5abebbe949049fccd1a684b5c279 P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 1.
I20260926 19:59:47.602696 15004 raft_consensus.cc:2468] T c86b5abebbe949049fccd1a684b5c279 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56196546dad349b9bd344c4e0b6a6874 in term 1.
I20260926 19:59:47.603694 14795 leader_election.cc:304] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56196546dad349b9bd344c4e0b6a6874, e3e2a17ea4c94765be709a7ec8af4c8a; no voters:
I20260926 19:59:47.604564 15732 raft_consensus.cc:2804] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:47.605001 15732 raft_consensus.cc:697] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [term 1 LEADER]: Becoming Leader. State: Replica: 56196546dad349b9bd344c4e0b6a6874, State: Running, Role: LEADER
I20260926 19:59:47.605854 15732 consensus_queue.cc:237] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
W20260926 19:59:47.675377 14795 connection.cc:573] server connection from 127.0.0.1:49434 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:47.675433 14938 connection.cc:573] server connection from 127.0.0.1:50906 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:47.676162 14940 connection.cc:573] client connection to 127.11.233.254:32801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:47.676693 15082 connection.cc:573] server connection from 127.0.0.1:48696 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:47.679934 14795 connection.cc:573] client connection to 127.11.233.254:32801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:47.680717 15085 connection.cc:573] client connection to 127.11.233.254:32801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:47.682753 15207 connection.cc:573] client connection to 127.11.233.254:32801 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 19:59:47.683626 15677 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 19:59:47.678297 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.11.233.254:32801 (local address 127.11.233.194:37661)
0926 19:59:47.679265 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0926 19:59:47.679310 (+ 45us) client_negotiation.cc:175] Beginning negotiation
0926 19:59:47.679579 (+ 269us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0926 19:59:47.682776 (+ 3197us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":359}
W20260926 19:59:47.684602 15207 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.11.233.254:32801)
W20260926 19:59:47.685277 15207 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.233.254:32801)
W20260926 19:59:47.685917 15207 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.11.233.254:32801)
W20260926 19:59:47.687213 15207 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.11.233.254:32801)
W20260926 19:59:47.692610 15676 negotiation.cc:336] Failed RPC negotiation. Trace:
0926 19:59:47.690594 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.11.233.254:32801 (local address 127.11.233.195:50883)
0926 19:59:47.691577 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0926 19:59:47.691783 (+ 206us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":388}
I20260926 19:59:47.732249 15757 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:47.732788 15757 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:47.737136 15757 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:47.739903 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:47.747368 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:47.748124 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:47.748472 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:47.749100 15757 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:47.750264 15729 raft_consensus.cc:493] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:59:47.750773 15729 raft_consensus.cc:515] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.752663 15729 leader_election.cc:290] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633), 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755)
I20260926 19:59:47.754022 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94b81663f0b14be39a3be2d13035f3f4" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874" is_pre_election: true
I20260926 19:59:47.754660 14860 raft_consensus.cc:2468] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 0.
I20260926 19:59:47.755676 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94b81663f0b14be39a3be2d13035f3f4" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" is_pre_election: true
I20260926 19:59:47.756320 15149 raft_consensus.cc:2468] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 0.
I20260926 19:59:47.757654 14940 leader_election.cc:304] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, e3e2a17ea4c94765be709a7ec8af4c8a; no voters:
I20260926 19:59:47.758452 15729 raft_consensus.cc:2804] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:59:47.758775 15729 raft_consensus.cc:493] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:59:47.759104 15729 raft_consensus.cc:3060] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.765828 15729 raft_consensus.cc:515] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:47.767747 15729 leader_election.cc:290] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 election: Requested vote from peers e3e2a17ea4c94765be709a7ec8af4c8a (127.11.233.195:46633), 56196546dad349b9bd344c4e0b6a6874 (127.11.233.193:40755)
I20260926 19:59:47.768524 15149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94b81663f0b14be39a3be2d13035f3f4" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a"
I20260926 19:59:47.768868 14860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94b81663f0b14be39a3be2d13035f3f4" candidate_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56196546dad349b9bd344c4e0b6a6874"
I20260926 19:59:47.769078 15149 raft_consensus.cc:3060] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.769426 14860 raft_consensus.cc:3060] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:59:47.775758 15149 raft_consensus.cc:2468] T 94b81663f0b14be39a3be2d13035f3f4 P e3e2a17ea4c94765be709a7ec8af4c8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 1.
I20260926 19:59:47.776131 14860 raft_consensus.cc:2468] T 94b81663f0b14be39a3be2d13035f3f4 P 56196546dad349b9bd344c4e0b6a6874 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17fec3df2f014ed5b1803b8a2bc38df9 in term 1.
I20260926 19:59:47.777130 14940 leader_election.cc:304] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17fec3df2f014ed5b1803b8a2bc38df9, e3e2a17ea4c94765be709a7ec8af4c8a; no voters:
I20260926 19:59:47.778110 15729 raft_consensus.cc:2804] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:59:47.779135 15729 raft_consensus.cc:697] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [term 1 LEADER]: Becoming Leader. State: Replica: 17fec3df2f014ed5b1803b8a2bc38df9, State: Running, Role: LEADER
I20260926 19:59:47.779917 15729 consensus_queue.cc:237] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 } } peers { permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 } } peers { permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 } }
I20260926 19:59:48.264137 15764 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:48.264564 15764 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:48.266453 15764 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:48.278906 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:48.279498 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:48.279824 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:48.280421 15764 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:48.374394 15769 consensus_queue.cc:1048] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260926 19:59:48.413920 15769 consensus_queue.cc:1048] T c86b5abebbe949049fccd1a684b5c279 P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:48.571698 15773 consensus_queue.cc:1048] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:48.592090 15773 consensus_queue.cc:1048] T 535d135b220c4855b687e7315aaaa399 P 56196546dad349b9bd344c4e0b6a6874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17fec3df2f014ed5b1803b8a2bc38df9" member_type: VOTER last_known_addr { host: "127.11.233.194" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:48.712416 15770 consensus_queue.cc:1048] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3e2a17ea4c94765be709a7ec8af4c8a" member_type: VOTER last_known_addr { host: "127.11.233.195" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:48.735816 15770 consensus_queue.cc:1048] T 94b81663f0b14be39a3be2d13035f3f4 P 17fec3df2f014ed5b1803b8a2bc38df9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56196546dad349b9bd344c4e0b6a6874" member_type: VOTER last_known_addr { host: "127.11.233.193" port: 40755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:59:48.960297 12199 master-stress-test.cc:579] Killing test thread 1/10
I20260926 19:59:49.151075 15789 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:49.151793 15789 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:49.153988 15789 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:49.170864 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:49.171495 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:49.171880 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:49.172574 15789 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:49.830000 15807 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:49.830436 15807 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:49.831945 15807 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:49.837402 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:49.838034 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:49.838338 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:49.838919 15807 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:50.851238 15830 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:50.851691 15830 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:50.853822 15830 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:50.859217 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:50.859819 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:50.860137 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:50.860802 15830 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:51.675779 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 19:59:51.676156 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:59:54.424188 15889 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:54.424780 15889 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:54.426946 15889 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:54.430523 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260926 19:59:54.434484 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:54.434940 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:54.435283 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:54.435882 15889 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 19:59:56.679725 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 19:59:56.680254 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:59:57.440198 15937 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 19:59:57.440645 15937 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 19:59:57.445190 15937 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 19:59:57.450474 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 19:59:57.451025 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 19:59:57.451336 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 19:59:57.451915 15937 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:00:01.682850 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:01.683235 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:05.877396 16065 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:00:05.878036 16065 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:00:05.880087 16065 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:00:05.883024 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 20:00:05.886030 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:00:05.886732 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:00:05.887032 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:00:05.887575 16065 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:00:06.686532 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:06.686954 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:08.108966 16101 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:00:08.109396 16101 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:00:08.111057 16101 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:00:08.116535 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:00:08.117266 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:00:08.117671 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:00:08.118357 16101 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 20:00:10.607435 15663 debug-util.cc:398] Leaking SignalData structure 0x7b080005a300 after lost signal to thread 15577
W20260926 20:00:10.608110 15663 debug-util.cc:398] Leaking SignalData structure 0x7b080007c940 after lost signal to thread 15666
I20260926 20:00:11.725430 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:11.725891 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:16.729856 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:16.730350 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:19.107865 16275 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:00:19.108394 16275 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:00:19.110209 16275 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:00:19.112665 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 20:00:19.116983 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:00:19.118392 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:00:19.118916 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:00:19.119926 16275 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:00:21.733369 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:21.733836 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:24.955364 16371 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:00:24.955994 16371 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:00:24.957746 16371 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:00:24.961050 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:00:24.965023 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:00:24.965785 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:00:24.966058 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:00:24.966625 16371 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:00:26.737056 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:26.737572 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:31.740274 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:31.740696 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:00:34.112605 15663 debug-util.cc:398] Leaking SignalData structure 0x7b080005abc0 after lost signal to thread 15577
W20260926 20:00:34.113111 15663 debug-util.cc:398] Leaking SignalData structure 0x7b0800070080 after lost signal to thread 15666
W20260926 20:00:36.265982 15052 heartbeater.cc:646] Failed to heartbeat to 127.11.233.254:32801 (105 consecutive failures): Network error: Failed to ping master at 127.11.233.254:32801: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:00:36.453989 14906 heartbeater.cc:646] Failed to heartbeat to 127.11.233.252:37917 (123 consecutive failures): Network error: Failed to ping master at 127.11.233.252:37917: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:00:36.575295 16549 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:00:36.575878 16549 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:00:36.577489 16549 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:00:36.580259 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:00:36.583600 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:00:36.584251 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:00:36.584510 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:00:36.585006 16549 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 20:00:36.591809 15196 heartbeater.cc:646] Failed to heartbeat to 127.11.233.254:32801 (103 consecutive failures): Network error: Failed to ping master at 127.11.233.254:32801: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 20:00:36.773473 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:36.773973 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:41.776742 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:41.777155 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:00:42.440363 15190 debug-util.cc:398] Leaking SignalData structure 0x7b080003e2a0 after lost signal to thread 15062
W20260926 20:00:42.441201 15190 debug-util.cc:398] Leaking SignalData structure 0x7b08000dab20 after lost signal to thread 15193
I20260926 20:00:46.781075 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:46.781575 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:51.784860 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:51.785252 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:00:52.489581 16793 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:00:52.490126 16793 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:00:52.492069 16793 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:00:52.494764 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:00:52.498067 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:00:52.498651 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:00:52.498890 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:00:52.499501 16793 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 20:00:55.810490 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd220 after lost signal to thread 14741
W20260926 20:00:55.812006 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000e08c0 after lost signal to thread 14904
I20260926 20:00:56.789017 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:00:56.789659 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:01:01.792433 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:01.792821 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:01:02.364846 15045 debug-util.cc:398] Leaking SignalData structure 0x7b080003d620 after lost signal to thread 14916
W20260926 20:01:02.366257 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1000 after lost signal to thread 15049
W20260926 20:01:04.798631 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000bd3c0 after lost signal to thread 14741
W20260926 20:01:04.799712 14901 debug-util.cc:398] Leaking SignalData structure 0x7b080006e0e0 after lost signal to thread 14904
I20260926 20:01:06.796514 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:06.797079 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:01:10.654917 17027 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:01:10.655421 17027 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:01:10.657290 17027 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:01:10.659838 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:01:10.664000 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:01:10.664659 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:10.664961 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:01:10.665596 17027 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:01:11.800086 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:11.800484 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:01:16.804052 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:16.804486 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:01:19.211822 17126 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:01:19.212307 17126 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:01:19.213920 17126 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:01:19.216895 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:01:19.219870 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:01:19.220623 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:19.220896 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:01:19.221406 17126 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:01:21.807392 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:21.807816 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:01:25.591367 14901 debug-util.cc:398] Leaking SignalData structure 0x7b0800042940 after lost signal to thread 14741
W20260926 20:01:25.592504 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3180 after lost signal to thread 14904
I20260926 20:01:26.811405 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:26.811862 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:01:27.954511 15663 debug-util.cc:398] Leaking SignalData structure 0x7b08000a02a0 after lost signal to thread 15577
W20260926 20:01:27.955044 15663 debug-util.cc:398] Leaking SignalData structure 0x7b080007c9e0 after lost signal to thread 15666
W20260926 20:01:29.030224 15663 debug-util.cc:398] Leaking SignalData structure 0x7b0800070800 after lost signal to thread 15577
W20260926 20:01:29.030792 15663 debug-util.cc:398] Leaking SignalData structure 0x7b0800070440 after lost signal to thread 15666
I20260926 20:01:31.859138 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:31.859647 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:01:36.461918 14906 heartbeater.cc:646] Failed to heartbeat to 127.11.233.252:37917 (242 consecutive failures): Network error: Failed to ping master at 127.11.233.252:37917: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 20:01:36.739709 15051 heartbeater.cc:646] Failed to heartbeat to 127.11.233.252:37917 (242 consecutive failures): Network error: Failed to ping master at 127.11.233.252:37917: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:36.863097 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:36.863610 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:01:37.072332 15195 heartbeater.cc:646] Failed to heartbeat to 127.11.233.252:37917 (240 consecutive failures): Network error: Failed to ping master at 127.11.233.252:37917: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:37.617854 17366 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:01:37.618393 17366 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:01:37.620446 17366 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:01:37.623256 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:01:37.626547 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:01:37.627242 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:37.627540 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:01:37.628125 17366 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:01:41.866205 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:41.866588 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:01:46.592075 15621 maintenance_manager.cc:419] P fa6dae08245e46c98e01465e84732c92: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033350
I20260926 20:01:46.668376 15620 maintenance_manager.cc:643] P fa6dae08245e46c98e01465e84732c92: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.075s user 0.032s sys 0.022s Metrics: {"bytes_written":12203,"cfile_init":1,"drs_written":1,"lbm_read_time_us":237,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2399,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":21,"thread_start_us":451,"threads_started":1,"wal-append.queue_time_us":757}
I20260926 20:01:46.670529 15621 maintenance_manager.cc:419] P fa6dae08245e46c98e01465e84732c92: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1309 bytes on disk
I20260926 20:01:46.671954 15620 maintenance_manager.cc:643] P fa6dae08245e46c98e01465e84732c92: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":211,"lbm_reads_lt_1ms":4}
I20260926 20:01:46.869951 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:46.870359 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:01:48.667177 17537 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:01:48.667732 17537 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:01:48.669485 17537 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:01:48.671833 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:01:48.674772 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:01:48.675320 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:48.675586 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:01:48.676178 17537 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:01:49.644325 17554 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:01:49.644773 17554 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:01:49.646549 17554 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:01:49.652285 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:01:49.652696 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:49.652923 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:01:49.653422 17554 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:01:51.872848 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:51.873255 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:01:56.876767 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:01:56.877269 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:01:58.183169 17705 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:01:58.183723 17705 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:01:58.185437 17705 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:01:58.187701 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 20:01:58.190528 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:01:58.191058 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:58.191340 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:01:58.191919 17705 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:01:59.413785 17731 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:01:59.414242 17731 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:01:59.415956 17731 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:01:59.421787 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:01:59.423875 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:01:59.424175 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:01:59.424675 17731 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 20:02:00.829908 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1860 after lost signal to thread 14741
W20260926 20:02:00.830777 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6460 after lost signal to thread 14904
I20260926 20:02:00.864823 17762 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:02:00.865285 17762 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:02:00.866998 17762 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:02:00.872006 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:02:00.872764 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:02:00.873031 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:02:00.873548 17762 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:02:01.880254 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:01.880614 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:06.884321 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:06.884932 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:02:09.285249 15190 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9f40 after lost signal to thread 15062
W20260926 20:02:09.286260 15190 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd0a0 after lost signal to thread 15193
W20260926 20:02:10.805562 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000ddf40 after lost signal to thread 14741
W20260926 20:02:10.806725 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000bca80 after lost signal to thread 14904
I20260926 20:02:11.888279 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:11.888638 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:14.488668 18023 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:02:14.489228 18023 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:02:14.490926 18023 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:02:14.493752 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260926 20:02:14.497089 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:02:14.497785 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:02:14.498164 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:02:14.498716 18023 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:02:16.892321 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:16.892913 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:21.895989 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:21.896412 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:26.899963 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:26.900450 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:31.903383 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:31.903805 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:32.439160 18365 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:02:32.439725 18365 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:02:32.441807 18365 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:02:32.444166 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:02:32.447151 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:02:32.447906 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:02:32.448175 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:02:32.448685 18365 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 20:02:36.892297 14909 heartbeater.cc:646] Failed to heartbeat to 127.11.233.254:32801 (342 consecutive failures): Network error: Failed to ping master at 127.11.233.254:32801: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 20:02:36.907465 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:36.907994 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:02:37.231135 15051 heartbeater.cc:646] Failed to heartbeat to 127.11.233.252:37917 (362 consecutive failures): Network error: Failed to ping master at 127.11.233.252:37917: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 20:02:37.526829 15195 heartbeater.cc:646] Failed to heartbeat to 127.11.233.252:37917 (360 consecutive failures): Network error: Failed to ping master at 127.11.233.252:37917: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:02:39.144936 18494 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:02:39.145498 18494 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:02:39.147325 18494 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:02:39.149639 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:02:39.152741 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:02:39.153529 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:02:39.153825 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:02:39.154403 18494 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:02:41.910584 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:41.910961 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:46.916123 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:46.916729 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:02:47.913125 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000d4960 after lost signal to thread 14916
W20260926 20:02:47.914120 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6ce0 after lost signal to thread 15049
I20260926 20:02:51.920403 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:51.920857 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:56.899082 18826 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:02:56.899968 18826 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:02:56.903108 18826 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:02:56.906543 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:02:56.909765 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:02:56.910495 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:02:56.910804 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:02:56.911393 18826 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:02:56.923206 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:02:56.923688 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:02:57.498811 18840 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:02:57.499238 18840 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:02:57.500742 18840 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:02:57.505923 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:02:57.506594 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:02:57.506894 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:02:57.507464 18840 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:03:01.927187 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:01.927668 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:03:04.226521 18966 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:03:04.226991 18966 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:03:04.228709 18966 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:03:04.231279 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260926 20:03:04.234174 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:03:04.234778 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:03:04.235095 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:03:04.235615 18966 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:03:06.931743 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:06.932296 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:03:10.653597 19088 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:03:10.654289 19088 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:03:10.656601 19088 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:03:10.664397 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:03:10.686538 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:03:10.687093 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:03:10.687357 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:03:10.687974 19088 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 20:03:10.960287 15663 debug-util.cc:398] Leaking SignalData structure 0x7b080008a4e0 after lost signal to thread 15577
W20260926 20:03:10.960837 15663 debug-util.cc:398] Leaking SignalData structure 0x7b0800070d20 after lost signal to thread 15666
W20260926 20:03:11.688772 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000de180 after lost signal to thread 14916
W20260926 20:03:11.689767 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7b60 after lost signal to thread 15049
I20260926 20:03:11.974082 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:11.974553 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:03:16.978703 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:16.979314 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:03:20.078187 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000840e0 after lost signal to thread 14916
W20260926 20:03:20.079169 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000425e0 after lost signal to thread 15049
I20260926 20:03:21.982579 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:21.983260 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:03:26.987735 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:26.988369 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:03:28.113152 19411 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:03:28.113703 19411 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:03:28.115892 19411 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:03:28.118655 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:03:28.121984 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:03:28.122696 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:03:28.122968 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:03:28.123509 19411 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:03:31.991446 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:31.991884 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:03:36.995987 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:36.996552 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:03:37.385823 14906 heartbeater.cc:646] Failed to heartbeat to 127.11.233.252:37917 (482 consecutive failures): Network error: Failed to ping master at 127.11.233.252:37917: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 20:03:37.722057 15051 heartbeater.cc:646] Failed to heartbeat to 127.11.233.252:37917 (482 consecutive failures): Network error: Failed to ping master at 127.11.233.252:37917: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 20:03:37.947080 15196 heartbeater.cc:646] Failed to heartbeat to 127.11.233.254:32801 (463 consecutive failures): Network error: Failed to ping master at 127.11.233.254:32801: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
I20260926 20:03:41.528326 19654 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:03:41.528852 19654 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:03:41.530747 19654 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:03:41.533255 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:03:41.536538 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:03:41.537304 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:03:41.537609 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:03:41.538218 19654 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 20:03:41.866109 15190 debug-util.cc:398] Leaking SignalData structure 0x7b08000dabe0 after lost signal to thread 15062
W20260926 20:03:41.866971 15190 debug-util.cc:398] Leaking SignalData structure 0x7b08000b8a20 after lost signal to thread 15193
I20260926 20:03:41.998919 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:41.999333 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:03:43.483425 15663 debug-util.cc:398] Leaking SignalData structure 0x7b0800080820 after lost signal to thread 15577
W20260926 20:03:43.484149 15663 debug-util.cc:398] Leaking SignalData structure 0x7b08000800a0 after lost signal to thread 15666
I20260926 20:03:47.025283 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:47.025911 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:03:52.028800 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:52.029275 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:03:55.264050 19853 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:03:55.264668 19853 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:03:55.266515 19853 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:03:55.269408 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:03:55.273074 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:03:55.273732 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:03:55.273976 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:03:55.274525 19853 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:03:57.032212 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:03:57.032621 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:04:02.035995 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:04:02.036365 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:04:07.040274 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:04:07.040910 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:04:09.947146 14901 debug-util.cc:398] Leaking SignalData structure 0x7b080003eba0 after lost signal to thread 14741
W20260926 20:04:09.948056 14901 debug-util.cc:398] Leaking SignalData structure 0x7b08000cf320 after lost signal to thread 14904
W20260926 20:04:10.254446 15663 debug-util.cc:398] Leaking SignalData structure 0x7b08000bae60 after lost signal to thread 15577
W20260926 20:04:10.255066 15663 debug-util.cc:398] Leaking SignalData structure 0x7b08000b2ae0 after lost signal to thread 15666
W20260926 20:04:10.741024 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4f20 after lost signal to thread 14916
W20260926 20:04:10.741971 15045 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5500 after lost signal to thread 15049
I20260926 20:04:12.069139 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:04:12.069514 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:04:13.252349 20105 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:04:13.252878 20105 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:04:13.254988 20105 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:04:13.257361 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:04:13.260483 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:04:13.261162 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:04:13.261422 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:04:13.262012 20105 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260926 20:04:17.073024 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:04:17.073547 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:04:18.849637 20182 raft_consensus.cc:493] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 26c8ef2c095f418083f33f8f0a734c3a)
I20260926 20:04:18.850148 20182 raft_consensus.cc:515] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c8ef2c095f418083f33f8f0a734c3a" member_type: VOTER last_known_addr { host: "127.11.233.254" port: 32801 } } peers { permanent_uuid: "fa6dae08245e46c98e01465e84732c92" member_type: VOTER last_known_addr { host: "127.11.233.253" port: 39245 } } peers { permanent_uuid: "eb0f126418d54df489a4ce95a765b8d3" member_type: VOTER last_known_addr { host: "127.11.233.252" port: 37917 } }
I20260926 20:04:18.851773 20182 leader_election.cc:290] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801), eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917)
W20260926 20:04:18.854327 15619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260926 20:04:18.857283 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c8ef2c095f418083f33f8f0a734c3a (127.11.233.254:32801): Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111)
W20260926 20:04:18.858095 15619 leader_election.cc:336] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer eb0f126418d54df489a4ce95a765b8d3 (127.11.233.252:37917): Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
I20260926 20:04:18.858474 15619 leader_election.cc:304] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa6dae08245e46c98e01465e84732c92; no voters: 26c8ef2c095f418083f33f8f0a734c3a, eb0f126418d54df489a4ce95a765b8d3
I20260926 20:04:18.859205 20182 raft_consensus.cc:2749] T 00000000000000000000000000000000 P fa6dae08245e46c98e01465e84732c92 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260926 20:04:21.455930 15663 debug-util.cc:398] Leaking SignalData structure 0x7b0800070360 after lost signal to thread 15577
W20260926 20:04:21.456513 15663 debug-util.cc:398] Leaking SignalData structure 0x7b080003c960 after lost signal to thread 15666
W20260926 20:04:21.849637 15190 debug-util.cc:398] Leaking SignalData structure 0x7b08000b66c0 after lost signal to thread 15062
W20260926 20:04:21.850575 15190 debug-util.cc:398] Leaking SignalData structure 0x7b08000846e0 after lost signal to thread 15193
I20260926 20:04:22.086017 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:04:22.086412 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:04:27.089958 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:04:27.090396 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:04:32.093382 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:04:32.093814 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:04:37.097261 15565 master_runner.cc:187] Error getting master registration for 127.11.233.254:32801: Network error: Client connection negotiation failed: client connection to 127.11.233.254:32801: connect: Connection refused (error 111),
I20260926 20:04:37.097869 15565 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:04:37.367982 15207 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917, num_attempts: 755) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 20:04:37.368547 15207 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917, num_attempts: 755) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 20:04:37.368818 15207 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917, num_attempts: 755) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
W20260926 20:04:37.369145 15207 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917, num_attempts: 755) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
F20260926 20:04:37.382578 15218 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917, num_attempts: 755) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20260926 20:04:37.382967 15216 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917, num_attempts: 755) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
*** Aborted at 1790453077 (unix time) try "date -d @1790453077" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** Check failure stack trace: ***
*** SIGABRT (@0x3e800002fa7) received by PID 12199 (TID 0x7f23894f9700) from PID 12199; stack trace: ***
@ 0x2b0620 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b2b03 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7f239de65980 (unknown) at ??:0
W20260926 20:04:37.412999 15207 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.11.233.254:32801)
W20260926 20:04:37.414624 15207 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.233.254:32801)
@ 0x7f239900fe87 gsignal at ??:0
@ 0x7f23990117f1 abort at ??:0
@ 0x2b6416 __interceptor_abort at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
F20260926 20:04:37.382967 15216 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917, num_attempts: 755) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)F20260926 20:04:37.406555 15217 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.11.233.254:32801,127.11.233.253:39245,127.11.233.252:37917, num_attempts: 755) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.11.233.252:37917: connect: Connection refused (error 111)
*** Check failure stack trace: ***
@ 0x7f239c0add74 google::LogMessage::SendToLog() at ??:0
==================
WARNING: ThreadSanitizer: data race (pid=12199)
Atomic write of size 8 at 0x7f239c2d98b8 by thread T32:
#0 __tsan_atomic64_compare_exchange_val /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interface_atomic.cpp:850 (master-stress-test+0x30a0c8)
#1 unsigned long* google::glog_internal_namespace_::sync_val_compare_and_swap<unsigned long*>(unsigned long**, unsigned long*, unsigned long*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/utilities.h:172:10 (libglog.so.1+0x2b255)
#2 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/signalhandler.cc:296:7 (libglog.so.1+0x2b255)
#3 __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1911 (master-stress-test+0x2b061f)
#4 google::LogMessage::Fail() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/logging.cc:1946:3 (libglog.so.1+0x18d73)
#5 google::LogMessage::SendToLog() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/logging.cc:1920:5 (libglog.so.1+0x18d73)
#6 google::LogMessage::Flush() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/logging.cc:1777:5 (libglog.so.1+0x1991f)
#7 google::LogMessageFatal::~LogMessageFatal() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/logging.cc:2557:5 (libglog.so.1+0x1ea5a)
#8 kudu::MasterStressTest::AlterTableThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:330:7 (master-stress-test+0x35ea35)
#9 decltype(*(std::__1::forward<kudu::MasterStressTest_Test_Test*>(fp0)).*fp()) std::__1::__invoke<void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*, void>(void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3846:1 (master-stress-test+0x38bc3d)
#10 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>&, std::__1::__tuple_indices<2ul>) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:280:5 (master-stress-test+0x38baf0)
#11 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*> >(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:291:5 (master-stress-test+0x38b23a)
Previous read of size 8 at 0x7f239c2d98b8 by thread T31:
#0 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/signalhandler.cc:302:38 (libglog.so.1+0x2ca60)
@ 0x7f239c0ae920 google::LogMessage::Flush() at ??:0
#1 __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1911 (master-stress-test+0x2b061f)
#2 google::LogMessage::Fail() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/logging.cc:1946:3 (libglog.so.1+0x18d73)
#3 google::LogMessage::SendToLog() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/logging.cc:1920:5 (libglog.so.1+0x18d73)
#4 google::LogMessage::Flush() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/logging.cc:1777:5 (libglog.so.1+0x1991f)
#5 google::LogMessageFatal::~LogMessageFatal() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/logging.cc:2557:5 (libglog.so.1+0x1ea5a)
#6 kudu::MasterStressTest::AlterTableThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:330:7 (master-stress-test+0x35ea35)
#7 decltype(*(std::__1::forward<kudu::MasterStressTest_Test_Test*>(fp0)).*fp()) std::__1::__invoke<void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*, void>(void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3846:1 (master-stress-test+0x38bc3d)
#8 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>&, std::__1::__tuple_indices<2ul>) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:280:5 (master-stress-test+0x38baf0)
#9 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*> >(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:291:5 (master-stress-test+0x38b23a)
Location is global 'google::(anonymous namespace)::g_entered_thread_id_pointer' of size 8 at 0x7f239c2d98b8 (libglog.so.1+0x0000002448b8)
Thread T32 (tid=15217, running) created by main thread at:
#0 pthread_create /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (master-stress-test+0x2b3f95)
#1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__threading_support:504:10 (master-stress-test+0x38b1cb)
#2 std::__1::thread::thread<void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*, void>(void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:307:16 (master-stress-test+0x38af27)
#3 void std::__1::allocator<std::__1::thread>::construct<std::__1::thread, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(std::__1::thread*, void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1809:31 (master-stress-test+0x38ae2c)
#4 void std::__1::allocator_traits<std::__1::allocator<std::__1::thread> >::__construct<std::__1::thread, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(std::__1::integral_constant<bool, true>, std::__1::allocator<std::__1::thread>&, std::__1::thread*, void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1687:21 (master-stress-test+0x38adc3)
#5 void std::__1::allocator_traits<std::__1::allocator<std::__1::thread> >::construct<std::__1::thread, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(std::__1::allocator<std::__1::thread>&, std::__1::thread*, void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1538:14 (master-stress-test+0x38ad03)
#6 void std::__1::vector<std::__1::thread, std::__1::allocator<std::__1::thread> >::__construct_one_at_end<void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:926:5 (master-stress-test+0x38aa0e)
#7 std::__1::thread& std::__1::vector<std::__1::thread, std::__1::allocator<std::__1::thread> >::emplace_back<void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:1687:9 (master-stress-test+0x35de9a)
#8 kudu::MasterStressTest_Test_Test::TestBody() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:562:13 (master-stress-test+0x35cd86)
#9 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64dff)
#10 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64dff)
#11 testing::Test::Run() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a01)
#12 testing::TestInfo::Run() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d18)
#13 testing::TestSuite::Run() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44cf4)
#14 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597e4)
#15 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65cbf)
#16 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65cbf)
#17 testing::UnitTest::Run() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58d9c)
#18 RUN_ALL_TESTS() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (master-stress-test+0x38e42b)
#19 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_main.cc:115:10 (master-stress-test+0x38d03c)
Thread T31 (tid=15216, running) created by main thread at:
#0 pthread_create /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (master-stress-test+0x2b3f95)
#1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__threading_support:504:10 (master-stress-test+0x38b1cb)
#2 std::__1::thread::thread<void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*, void>(void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:307:16 (master-stress-test+0x38af27)
#3 void std::__1::allocator<std::__1::thread>::construct<std::__1::thread, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(std::__1::thread*, void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1809:31 (master-stress-test+0x38ae2c)
#4 void std::__1::allocator_traits<std::__1::allocator<std::__1::thread> >::__construct<std::__1::thread, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(std::__1::integral_constant<bool, true>, std::__1::allocator<std::__1::thread>&, std::__1::thread*, void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1687:21 (master-stress-test+0x38adc3)
#5 void std::__1::allocator_traits<std::__1::allocator<std::__1::thread> >::construct<std::__1::thread, void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(std::__1::allocator<std::__1::thread>&, std::__1::thread*, void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1538:14 (master-stress-test+0x38ad03)
#6 void std::__1::vector<std::__1::thread, std::__1::allocator<std::__1::thread> >::__emplace_back_slow_path<void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:1670:5 (master-stress-test+0x38ab3e)
#7 std::__1::thread& std::__1::vector<std::__1::thread, std::__1::allocator<std::__1::thread> >::emplace_back<void (kudu::MasterStressTest::*)(), kudu::MasterStressTest_Test_Test*>(void (kudu::MasterStressTest::*&&)(), kudu::MasterStressTest_Test_Test*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:1690:9 (master-stress-test+0x35deaa)
#8 kudu::MasterStressTest_Test_Test::TestBody() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:562:13 (master-stress-test+0x35cd86)
#9 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64dff)
#10 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64dff)
#11 testing::Test::Run() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a01)
#12 testing::TestInfo::Run() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d18)
#13 testing::TestSuite::Run() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44cf4)
#14 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x597e4)
#15 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65cbf)
#16 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65cbf)
#17 testing::UnitTest::Run() /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58d9c)
#18 RUN_ALL_TESTS() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (master-stress-test+0x38e42b)
#19 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_main.cc:115:10 (master-stress-test+0x38d03c)
SUMMARY: ThreadSanitizer: data race /home/jenkins-slave/workspace/build_and_test@2/thirdparty/src/glog-0.6.0/src/utilities.h:172:10 in unsigned long* google::glog_internal_namespace_::sync_val_compare_and_swap<unsigned long*>(unsigned long**, unsigned long*, unsigned long*)
==================
@ 0x7f239c0b3a5b google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x35ec5e kudu::MasterStressTest::DeleteTableThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x38bc3e _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3846
@ 0x38baf1 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x38b23b std::__1::__thread_proxy<>() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b04e0 __tsan_thread_start_func at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7f239de5a6db start_thread at ??:0
@ 0x7f23990f261f clone at ??:0